Re: [HACKERS] Timing overhead and Linux clock sources

2012-08-28 Thread Ants Aasma
to say that the histogram is mostly useless. It seems to me that it mostly shows OS scheduling noise. I would even say that the histogram output should be hidden behind an command line option to avoid unnecessary confusion. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener

Re: [HACKERS] SP-GiST micro-optimizations

2012-08-28 Thread Ants Aasma
overlap with other non-store instructions. [1] http://support.amd.com/us/Processor_TechDocs/25112.PDF [2] http://www.intel.com/content/dam/doc/manual/64-ia-32-architectures-optimization-manual.pdf Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http

Re: [HACKERS] [PATCH] Lazy hashaggregate when no aggregation is needed

2012-07-02 Thread Ants Aasma
marking this patch Rejected. Thank you, for considering this and many thanks to Etsuro Fujita for reviewing. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] [PATCH] Lazy hashaggregate when no aggregation is needed

2012-06-27 Thread Ants Aasma
. For me this was mostly a learning experience for poking around in the planner. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] [PATCH] Lazy hashaggregate when no aggregation is needed

2012-06-15 Thread Ants Aasma
. It won't help set returning functions because the tuplestore for those is fully materialized when the first row is fetched. [1] http://archives.postgresql.org/message-id/16737833.463.1332881676120.JavaMail.geo-discussion-forums%40pbcpw7 Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-06 Thread Ants Aasma
at 90tps with the stated configuration. Even when upping the shared_buffers and enabling indexonlyscan I didn't see more than about 540tps per thread. The test is designed to exercise buffer eviction, doing about 9800 buffer reads per transaction with 32MB of buffers. Ants Aasma -- Cybertec Schönig

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-04 Thread Ants Aasma
and doesn't even aspire to be portable yet. The main point was to see if there's any significant performance to be gained by this method. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Ants Aasma
in the docs that would even hint that using -x shouldn't work to create a replica. Why does it get confused and can we (easily) make it not get confused? At the very least it needs a big fat warning in documentation for the -x option that the resulting backup might not be usable as a standby. Ants Aasma

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-04 Thread Ants Aasma
? It was dead code as far as I could tell. That change isn't actually relevant for this patch because free-list management is still protected by a lock (except the initial unlocked test that is doublechecked under lock) and so doesn't need any adjustment. Ants Aasma -- Cybertec Schönig Schönig GmbH

Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Ants Aasma
On Mon, Jun 4, 2012 at 6:20 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Jun 4, 2012 at 11:25 PM, Ants Aasma a...@cybertec.at wrote: On Thu, Sep 29, 2011 at 11:30 PM, Magnus Hagander mag...@hagander.net wrote: it doesn't say that is not possible to use this for a standby server

Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Ants Aasma
this. I'll let you know if I find out how I managed to create this error. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Ants Aasma
to see where that train of thought takes me. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Ants Aasma
be a larger problem and post a higher gain. Either way, I think the nailing approach should be explored further, cacheline ping-pong could still be a problem with higher number of processors and losing the spinlock also loses the ability to detect contention. Ants Aasma -- Cybertec Schönig Schönig

Re: [HACKERS] slow dropping of tables, DropRelFileNodeBuffers, tas

2012-05-30 Thread Ants Aasma
in triggering the prefetchers. The CPU should be doing a lot better than the current ~4.3GB/s when scanning buffer descriptors. Of course not scanning at all or doing less scans at the expense of more work in the inner loop would be even better. Regards, Ants Aasma -- Cybertec Schönig Schönig GmbH

[HACKERS] Early hint bit setting

2012-05-30 Thread Ants Aasma
the transaction has been made visible. When someone flushes xlog, they also check if it enables some background hinting and set the corresponding flag for any backend with spare cycles to pick up. Comments? Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web

Re: [HACKERS] Early hint bit setting

2012-05-30 Thread Ants Aasma
. The latency vs throughput tradeoff could possibly be per backend tunable. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-24 Thread Ants Aasma
On Tue, May 22, 2012 at 11:36 PM, Ants Aasma a...@cybertec.at wrote: ... The free list itself is a bit trickier, but if it's still necessary/useful then SC-firstFreeBuffer and buf-freeNext are in effect a linked-list stack, there should plenty of tested lock free algorithms floating around

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-22 Thread Ants Aasma
://www.almaden.ibm.com/cs/people/dmodha/ARC.pdf Cheers, Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Ants Aasma
SELECT MIN(x) FROM test WHERE value BETWEEN :rangemin AND :rangemax; I get the following results: bitmap scan: 106 tps index scan: 146 tps index only scan: 653 tps Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-17 Thread Ants Aasma
660ns timing overhead and 26.5s execution for your query, with timing off execution time falls to 2.1s. For reference, tsc based timing gives 19.2ns overhead and 2.3s execution time with timing. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http

Re: [HACKERS] Strange issues with 9.2 pg_basebackup replication

2012-05-17 Thread Ants Aasma
write. 4) Then ... segfault! I cannot reproduce this. Attached is the script that I use for cascade replication testing. With it I can see the replica connecting to itself but no segfault. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http

Re: [HACKERS] master and sync-replica diverging

2012-05-17 Thread Ants Aasma
- replica_wal_sync - master_commit_visible - commit_response replica_wal_sync - replica_replay_wal - replica_commit_visible If you issue a select on the replica after getting a commit response from master you can see that the query getting a snapshot races with replay of the commit record. Ants Aasma

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-05-10 Thread Ants Aasma
the bias. But that definitely isn't in the territory of simple and would require rigorous statistical analysis. And as for the monetary unit sampling, I agree that this is better left as an optional extra. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-24 Thread Ants Aasma
On Tue, Apr 24, 2012 at 10:31 AM, Sandro Santilli s...@keybit.net wrote: On Tue, Apr 24, 2012 at 08:49:26AM +0200, Sandro Santilli wrote: On Mon, Apr 23, 2012 at 08:34:44PM +0300, Ants Aasma wrote: SELECT (SELECT reservoir_sample(some_table, 50) AS samples    FROM some_table WHERE ctid

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-23 Thread Ants Aasma
available users get more flexibility. The downside would be that we can't automatically make better sampling methods available. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Ants Aasma
://utopia.duth.gr/~pefraimi/research/data/2007EncOfAlg.pdf Cheers, Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Ants Aasma
be the minority anyway. I won't really miss the per table stats. But if the pg_stat_statements normalisation patch gets commited, it would be really neat to also have IO waits there. It would be much easier to quickly diagnose what is causing all this IO issues. Thanks again, Ants Aasma -- Cybertec Schönig

[HACKERS] [PATCH] Lazy hashaggregate when no aggregation is needed

2012-03-27 Thread Ants Aasma
aggregation. Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index c9aa921..53cdd09 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor

[HACKERS] pg_upgrade incorrectly equates pg_default and database tablespace

2012-03-25 Thread Ants Aasma
spclocation with database tablespace: tblspace = PQgetvalue(res, relnum, i_spclocation); /* if no table tablespace, use the database tablespace */ if (strlen(tblspace) == 0) tblspace = dbinfo-db_tblspace; Patch to fix this is attached. Regards, Ants Aasma -- Cybertec Schönig Schönig GmbH

Re: [HACKERS] pg_upgrade and statistics

2012-03-16 Thread Ants Aasma
-dimensional case and so people will want to try different algorithms, or make different tradeoffs on effort spent on constructing the histogram. Or even build one by hand. Cheers, Ants Aasma -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] performance results on IBM POWER7

2012-03-01 Thread Ants Aasma
/available_clocksource To switch the clocksource, just write the desired clocksource like this: # echo hpet /sys/devices/system/clocksource/clocksource0/current_clocksource Thanks, Ants Aasma -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Initial 9.2 pgbench write results

2012-02-27 Thread Ants Aasma
. Is there a standard benchmark to measure that? -- Ants Aasma

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-02-22 Thread Ants Aasma
some field experience on any issues surrounding timing :) Some implementation notes.  This currently fails regression test create_function_3, haven't looked into why yet. I'll take a look at it. Thanks again. -- Ants Aasma -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] CLOG contention, part 2

2012-02-10 Thread Ants Aasma
size. The increase in memory usage should be a drop in the bucket for systems that have enough transaction processing velocity for that to be a problem. -- Ants Aasma

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-21 Thread Ants Aasma
. -- Ants Aasma

Re: [HACKERS] 16-bit page checksums for 9.2

2012-01-04 Thread Ants Aasma
On Wed, Jan 4, 2012 at 3:49 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Dec 30, 2011 at 11:58 AM, Jeff Janes jeff.ja...@gmail.com wrote: On 12/29/11, Ants Aasma ants.aa...@eesti.ee wrote: Unless I'm missing something, double-writes are needed for all writes, not only the first page

Re: [HACKERS] 16-bit page checksums for 9.2

2011-12-29 Thread Ants Aasma
comes along, page A is broken in the heap with no double-write buffer backup nor anything to recover it by in the WAL. -- Ants Aasma -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Timing overhead and Linux clock sources

2011-12-09 Thread Ants Aasma
the variability will make the result pretty much useless. For example in the I/O case, a pretty typical load can have 1% of timings be 3 orders of magnitude longer than median. -- Ants Aasma #include stdio.h #include sys/time.h typedef long long int64; typedef unsigned long long uint64; typedef long int32

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-29 Thread Ants Aasma
is a patch, if anyone wishes to give it a go. -- Ants Aasma diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index 19ff524..59ff996 100644 --- a/src/backend/storage/ipc/procarray.c +++ b/src/backend/storage/ipc/procarray.c @@ -1324,7 +1324,7 @@ GetSnapshotData

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Ants Aasma
and avoid issues with someone changing the system time while you're timing. This precision does require OS and hardware cooperation, because of CPU offsets, TSC's changing frequencies, stopping, etc. -- Ants Aasma [1] https://github.com/torvalds/linux/blob/master/arch/x86/kernel/tsc_sync.c#L143

Re: [HACKERS] strict aliasing

2011-11-15 Thread Ants Aasma
improvement in generated code quality. -- Ants Aasma -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] concurrent snapshots

2011-09-08 Thread Ants Aasma
that they don’t change any of the basic characteristics. Your thoughts? -- Ants Aasma -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] concurrent snapshots

2011-09-08 Thread Ants Aasma
On Thu, Sep 8, 2011 at 5:28 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Sep 8, 2011 at 9:26 AM, Ants Aasma ants.aa...@eesti.ee wrote: When go try to find the new csnmin and discover that a backend has a csnmin that is too old, we go through the snapshots of that backend and convert

Re: [HACKERS] concurrent snapshots

2011-09-08 Thread Ants Aasma
in, but in hindsight seems a bit too much for a first try. -- Ants Aasma -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] spinlocks on HP-UX

2011-08-29 Thread Ants Aasma
to specs it should take 4 or 8GB DIMMs in pairs. Sounds like the server is split into multiple partitions. -- Ants Aasma [1] http://www.redbooks.ibm.com/redpapers/pdfs/redp4655.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] cheaper snapshots

2011-07-28 Thread Ants Aasma
be an optional WAL level on top of hot_standby that would only be enabled if consistent visibility on slaves is desired. -- Ants Aasma -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] cheaper snapshots

2011-07-28 Thread Ants Aasma
variant would say that if the commit returns, and the server doesn't crash in the meantime, the commit would at some point become visible. Maybe even that transactions that begin after the commit returns become visible after that commit. -- Ants Aasma -- Sent via pgsql-hackers mailing list (pgsql

<    1   2   3