Re: [HACKERS] IPv6 link-local addresses and init data type

2016-06-28 Thread Markus Wanner
;m starting to question if it's really wise to add the scope id to the INET6 type... > [2] - > http://stackoverflow.com/questions/24932172/what-length-can-a-network-interface-name-have Note that the scope id doesn't necessarily have to be a network interface name. Concluding there&#x

Re: [HACKERS] IPv6 link-local addresses and init data type

2016-06-03 Thread Markus Wanner
ugh. TEXT would be just fine to store a textual scope id. And it makes it utterly clear that there's no magic involved. Kind Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] IPv6 link-local addresses and init data type

2016-06-03 Thread Markus Wanner
On 06/03/2016 12:14 AM, Tom Lane wrote: > Markus Wanner writes: >> I'm even wondering if 'fe80::1%1'::inet = 'fe80::1%2'::inet shouldn't >> simply yield true. After all, it's the same (non-global) address. > > Surely not? If the zone_ids

Re: [HACKERS] IPv6 link-local addresses and init data type

2016-06-02 Thread Markus Wanner
obal addresses, I'm wondering what the use case for storing them is. I'm even wondering if 'fe80::1%1'::inet = 'fe80::1%2'::inet shouldn't simply yield true. After all, it's the same (non-global) address. Kind Regards Markus Wanner -- Sent via pgsql-hac

Re: [HACKERS] Proposal: template-ify (binary) extensions

2013-07-23 Thread Markus Wanner
g and providing examples. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: template-ify (binary) extensions

2013-07-22 Thread Markus Wanner
Dimitri, On 07/22/2013 08:44 PM, Dimitri Fontaine wrote: > That's the trade-off we currently need to make to be able to ship with > the current security protections we're talking about. Anything wrong with shipping postgis-1.5.so and postgis-2.0.so, as I we for Debian? > Ok, here's the full work

Re: [HACKERS] Proposal: template-ify (binary) extensions

2013-07-22 Thread Markus Wanner
y the case with a "security conscious sysadmin" - they very > seldom want to install anything. Exactly. That's why I'm favoring solutions that don't require any extension and keep the guarantee of preventing arbitrary native code. Regards Markus Wanner -

Re: [HACKERS] Proposal: template-ify (binary) extensions

2013-07-21 Thread Markus Wanner
Salut Dimitri, On 07/20/2013 01:23 AM, Dimitri Fontaine wrote: > Markus Wanner writes: >>> - per-installation (not even per-cluster) DSO availability >>> >>> If you install PostGIS 1.5 on a system, then it's just impossible to >>> bring a

Re: [HACKERS] Proposal: template-ify (binary) extensions

2013-07-17 Thread Markus Wanner
t on the server's, but only on the (libpq) client's file-system. No threat to the server. > Yes it's dangerous. It's also solving real world problems that I see no > other way to solve apart from bypassing the need to ever load a DSO > file, that is embedding a retargetable C compiler in the backend. If the sysadmin wants to disallow arbitrary execution of native code to postgres (the process), any kind of embedded compiler likely is equally unwelcome. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: template-ify (binary) extensions

2013-07-16 Thread Markus Wanner
quite so bad if we write the bits to a file first and > then dynamically load the file. That way at least noexec or similar > can provide protection. But it still seems like a pretty dangerous > direction. I agree now. Thanks for elaborating. Regards Markus Wanner --

Re: [HACKERS] Proposal: template-ify (binary) extensions

2013-07-15 Thread Markus Wanner
Robert, On 07/15/2013 12:12 PM, Markus Wanner wrote: > On 07/15/2013 05:49 AM, Robert Haas wrote (in a slightly different order): >> There is a lot of >> (well-founded, IMHO) resistance to the idea of letting users install C >> code via an SQL connection. > > Nowhere d

Re: [HACKERS] Proposal: template-ify (binary) extensions

2013-07-15 Thread Markus Wanner
On 07/15/2013 12:19 PM, Andres Freund wrote: > On 2013-07-15 12:12:36 +0200, Markus Wanner wrote: >> Granted, the "internalization" of the DSO is somewhat critical to >> implement. Running as a non-root user, Postgres has less power than that >> and can certainly n

Re: [HACKERS] Proposal: template-ify (binary) extensions

2013-07-15 Thread Markus Wanner
ave the power. > Now, I don't think Markus proposal is a good idea on *other* grounds > though... but that's for another email. Separation of concerns, huh? Good thing, thanks :-) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql

Re: [HACKERS] Proposal: template-ify (binary) extensions

2013-07-15 Thread Markus Wanner
bly check the permissions on the extension directory and at least emit a warning, if it's world-writable. Or refuse to create (or even load) an extension, right away. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: extension template

2013-07-10 Thread Markus Wanner
gs that live entirely inside the database" do you have in mind. > There would be value to inside-the-database package management, but it > should be a separate concept. Anything that's incompatible to extensions is not gonna fly. There are too many of them available, already. We need

Re: [HACKERS] Review: extension template

2013-07-09 Thread Markus Wanner
Salut Dimitri, On 07/09/2013 12:40 PM, Dimitri Fontaine wrote: > Markus Wanner writes: >> Or how do you think would pg_restore fail, if you followed the mental >> model of the template? > > # create template for extension foo version 'x' as ''; >

Re: [HACKERS] Review: extension template

2013-07-09 Thread Markus Wanner
te model). Another thing that surprised me is the inability to have an upgrade script *and* a full version (for the same extension target version). Even if that's intended behavior, the error message could be improved: > db1=# CREATE TEMPLATE FOR EXTENSION foo VERSION '0.0' AS

Re: [HACKERS] Millisecond-precision connect_timeout for libpq

2013-07-09 Thread Markus Wanner
usec is unsigned. (And the cited commit above indicates there are such platforms.) On 07/09/2013 02:25 PM, ivan babrou wrote: > There's no complexity here :) Not so fast, cowboy... :-) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Millisecond-precision connect_timeout for libpq

2013-07-09 Thread Markus Wanner
And if you're setting timeouts that low, you probably know what you're doing (or at least care about latency a lot). Or is gettimeofday() still considerably slower on certain architectures or in certain scenarios? Where's the complexity? Regards Markus Wanner -- Sent via pgsql-hackers

Re: [HACKERS] Millisecond-precision connect_timeout for libpq

2013-07-09 Thread Markus Wanner
On 07/08/2013 08:31 PM, ivan babrou wrote: > Seriously, I don't get why running 150 poolers is easier. Did you consider running pgbouncer on the database servers? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subs

Re: [HACKERS] Review: extension template

2013-07-08 Thread Markus Wanner
Hello Dimitri, On 07/08/2013 11:49 AM, Dimitri Fontaine wrote: > Please find attached to this mail version 9 of the Extension Templates > patch with fixes for the review up to now. Thanks, cool. > Markus Wanner writes: >>> I still think that we shouldn't allow cre

Re: [HACKERS] Review: extension template

2013-07-08 Thread Markus Wanner
te > the extensions your dump depends on in that new database, now pg_restore > your backup manually filtering away the extensions' objects or ignoring > the errors when pg_restore tries to duplicate functions you already > installed in the previous step. No fun. Definitel

Re: [HACKERS] Review: extension template

2013-07-08 Thread Markus Wanner
s why "template" is a misnomer (with the proposed implementation). "Extension" is not. (I still think "template" would be a good mental model. See my other thread... http://archives.postgresql.org/message-id/51d72c1d.7010...@bluegap.ch) Regards Markus Wanner -- Sent

Re: [HACKERS] Review: extension template

2013-07-07 Thread Markus Wanner
Hello Dimitri, On 07/07/2013 09:51 PM, Dimitri Fontaine wrote: > Markus Wanner writes: >> Oh, I just realize that pg_extension_{template,control,uptmpl} are not >> SHARED catalogs, but you need to install the template per-database and >> then need to enable it - per-database

Re: [HACKERS] Review: extension template

2013-07-07 Thread Markus Wanner
On 07/07/2013 02:55 PM, Markus Wanner wrote: > If you want to just upload extensions to a database via libpq.. Let's rephrase this in a (hopefully) more constructive way: I get the impression you are trying to satisfy many different needs. Way more that you need to scratch your own itch

Re: [HACKERS] Review: extension template

2013-07-07 Thread Markus Wanner
t;templates", those should be applicable to all databases, no? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: extension template

2013-07-07 Thread Markus Wanner
h. ] > It's just something I forgot to cleanup when completing the feature set. > Cleaned up in my git branch. Great! >> src/backend/commands/event_trigger.c, definition of >> event_trigger_support: several unnecessary whitespaces added. These make >> it hard(er) than

Re: [HACKERS] Review: extension template

2013-07-05 Thread Markus Wanner
On 07/05/2013 09:05 PM, Markus Wanner wrote: > The patch has already > been marked as 'returned with feedback', and I can support that > resolution (for this CF). Oops.. I just realize it's only set to "waiting on author", now. I guess I confused the two states.

[HACKERS] Proposal: template-ify (binary) extensions

2013-07-05 Thread Markus Wanner
ind shift for how binary extensions work. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Review: extension template

2013-07-05 Thread Markus Wanner
e. I didn't look too deep into the code, but it seems Jaime and Hitoshi raised some valid points. Assorted very minor nit-picks: In catalog/objectaddress.c, get_object_address_unqualified(): the case OBJECT_TABLESPACE is being moved down. That looks like an like an unintended change. Please rev

Re: [HACKERS] Hash partitioning.

2013-06-27 Thread Markus Wanner
don't know the total amount of work or the size of each piece in advance, it gets a bit harder. Choosing chunks that turn out to be too big certainly hurts. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Hash partitioning.

2013-06-27 Thread Markus Wanner
A single index performs better in that case, because it has lower overhead. I take your point that in case you *can* define a hot partition and apply partitioning, the hot(test) index(es) are more likely to be cached and thus require less disk I/O. Regards Markus Wanner -- Sent via pgs

Re: [HACKERS] Hash partitioning.

2013-06-27 Thread Markus Wanner
ets pretty close, I think. Though, I don't consider this to be related to how the tuples of the relation are laid out on disk. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Markus Wanner
o a normal, fully sequential scan and only fan out after the scan and distribute the incoming pages (or even tuples) to the multiple cores to process. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://w

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Markus Wanner
nor quality work with a "prize". Maybe a price for the best reviewer per CF? Maybe even based on votes from the general public on who's been the best, so reviews gain attention that way... "Click here to vote for my review." ... Maybe a crazy idea. Regards Markus Wanner

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Markus Wanner
urpose of parallel query execution, yes. But I see no reason for that not to work equally well for unpartitioned tables. Disk I/O is already pretty well optimized and parallelized, I think. Trying to parallelize a seq scan on the Postgres side is likely to yield far inferior performance. Regards Mark

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Markus Wanner
on large portions of a relation, IMO. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Markus Wanner
imization is one. (It's more like giving the system a hint. And we all dislike hints, don't we? *ducks*) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dynamic background workers

2013-06-20 Thread Markus Wanner
mple room for paranoia about postmaster interaction > with shared memory, but all it's doing is setting a flag, which is no > different from what CheckPostmasterSignal() already does. Sounds good to me. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dynamic background workers

2013-06-20 Thread Markus Wanner
n of the new > interface. With this patch, you can CREATE EXTENSION worker_spi and > then call worker_spi_launch(int4) to launch a new background worker, > or combine it with generate_series() to launch a bunch at once. Then > you can kill them off with pg_terminate_backend() and start some new

Re: [HACKERS] Change authentication error message (patch)

2013-06-20 Thread Markus Wanner
don't we also have to include "auth server unreachable" as a possible cause for authentication failure for methods that use an external server? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Change authentication error message (patch)

2013-06-19 Thread Markus Wanner
ht be hard to figure out which > row in pg_hba.conf was the one that PostgreSQL glommed onto to use for > authentication. As argued before, that should go into the logs for diagnosis by the sysadmin, but should not be revealed to an attacker. Regards Markus Wanner -- Sent via pgsql-hacker

Re: [HACKERS] Change authentication error message (patch)

2013-06-19 Thread Markus Wanner
covers any combination of a username/password > being wrong and obviously password expired covers the other. Works for me. Considering the password to be the thing that expires (rather than the account) is probably more accurate as well. Regards Markus Wanner -- Sent via pgsql-hackers mail

Re: [HACKERS] Change authentication error message (patch)

2013-06-18 Thread Markus Wanner
x27;s a bit longer, but sounds more like a full sentence, no? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal for CSN based snapshots

2013-06-10 Thread Markus Wanner
Ants, the more I think about this, the more I start to like it. On 06/07/2013 02:50 PM, Ants Aasma wrote: > On Fri, Jun 7, 2013 at 2:59 PM, Markus Wanner wrote: >> Agreed. Postgres-R uses a CommitOrderId, which is very similar in >> concept, for example. > > Do you think

Re: [HACKERS] Proposal for CSN based snapshots

2013-06-07 Thread Markus Wanner
thin that ring buffer. Where as currently, the snapshot's xip array resides in process-local memory. (Granted, often enough, the proc array also is a point of contention.) > At this point I don't see any major issues with this approach. If the > ensuing discussion doesn&#x

Re: [HACKERS] Re: logical changeset generation v3 - comparison to Postgres-R change set format

2013-01-14 Thread Markus Wanner
gt; > postgres=# SELECT length(textsend(NULL)); > length > > > (1 row) > > postgres=# SELECT length(textsend(NULL) || '\000'::bytea); > length > > > (1 row) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: logical changeset generation v3 - comparison to Postgres-R change set format

2013-01-13 Thread Markus Wanner
ess" operator could then be implemented by simply comparing two value's send() outputs. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-04 Thread Markus Wanner
On 12/03/2012 10:35 PM, Alvaro Herrera wrote: > So here's version 8. This fixes a couple of bugs and most notably > creates a separate PGPROC list for bgworkers, so that they don't > interfere with client-connected backends. Nice, thanks. I'll try to review this again, s

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-03 Thread Markus Wanner
t likely later this week, > or early next week; I would like to commit this patch before that. When > I'm back we can discuss other improvements. That will give us several > weeks before the end of the 9.3 development period to get these in. Of > course, other committers are we

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-03 Thread Markus Wanner
x27;s a net loss in robustness of the autovac implementation. Agreed. That's only one aspect of it, though. From the other point of view, it would be a proof of stability for the bgworker implementation if autovacuum worked on top of it. Regards Markus Wanner -- Sent via pgsql-hackers mail

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-03 Thread Markus Wanner
egards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-03 Thread Markus Wanner
out that guc.c obtains the > count of workers before workers have actually registered. So this > necessitates some reshuffling. Okay, thanks. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
se duration wants to > be tied with duration of postmaster, its author can consider to implement > it as background worker. I personally don't count that as a real need. There are better tools for this job; while there clearly are dangers in (ab)using the postmaster to do it. Regar

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
ostmaster's duration is a strong indication that it better not be a bgworker, I think. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
s long as things like pgbouncer, pgqd, etc.. keep to be available as separate daemons, I'm happy, though. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
us stuff separate (and advice developers to do so as well, instead of offering them a foot gun). So that our postmaster can do its job. And do it reliably, without trying to be a general purpose start/stop daemon. There are better and well established tools for that. Regards Markus Wanner -- Sent

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
libpq. But so can external processes. So that's no benefit of running under the postmaster. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
On 11/30/2012 11:09 AM, Dimitri Fontaine wrote: > Markus Wanner writes: >>> However, as you say, maybe we need more coding examples. >> >> Maybe a minimally usable extra daemon example? Showing how to avoid >> common pitfalls? Use cases, anybody? :-) >

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-28 Thread Markus Wanner
its log line upon request, i.e. show how to use SIGUSR1. > > KaiGai proposed that we remove auth_counter. I don't see why not; I > mean, worker_spi is already doing most of what auth_counter is doing, so > why not? Agreed. > However, as you say, maybe we need more codi

[HACKERS] Review: Extra Daemons / bgworker

2012-11-28 Thread Markus Wanner
hing and the kitchen sink.) # Tests No additional automated tests are included - hard to see how that could be tested automatically, given the lack of a proper test harness. I hope this review provides useful input. Regards Markus Wanner [1]: That was during commit fest 2010-09. Back then,

[HACKERS] Re: [pgsql-cluster-hackers] Question: Can i cut NON-HOT chain Pointers if there are no concurrent updates?

2012-11-23 Thread Markus Wanner
ter thesises befor me. Care to elaborate a bit? Can (part of) that code be released under an open license? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] logical changeset generation v3 - comparison to Postgres-R change set format

2012-11-17 Thread Markus Wanner
Hannu, On 11/17/2012 03:40 PM, Hannu Krosing wrote: > On 11/17/2012 03:00 PM, Markus Wanner wrote: >> On 11/17/2012 02:30 PM, Hannu Krosing wrote: >>> Is it possible to replicate UPDATEs and DELETEs without a primary key in >>> PostgreSQL-R >> No. There must be s

Re: [HACKERS] logical changeset generation v3 - comparison to Postgres-R change set format

2012-11-17 Thread Markus Wanner
KEY without breaking legacy applications that rely on SELECT * not returning that primary key. Are there other reasons to want tables without primary keys that I'm missing? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] logical changeset generation v3 - comparison to Postgres-R change set format

2012-11-17 Thread Markus Wanner
ary key without problems. It's the xmin of the old tuple that Postgres-R needs to get the COID. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] logical changeset generation v3 - comparison to Postgres-R change set format

2012-11-17 Thread Markus Wanner
Postgres >> nodes, where as the proposed patches are kept more general. > > One major aim definitely was optionally be able to replicate into just > about any target system, so yes, I certainly agree. I'm glad I got that correct ;-) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] logical changeset generation v3 - comparison to Postgres-R change set format

2012-11-16 Thread Markus Wanner
ummary, I'd say that Postgres-R is an approach specifically targeting and optimized for multi-master replication between Postgres nodes, where as the proposed patches are kept more general. I hope you found this to be an insightful and fair comparison. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-13 Thread Markus Wanner
outline an alternative that I think is viable and less intrusive. > This is why I think any good solution to this problem needs to > incorporate restartable conversion. I fully agree to that. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Markus Wanner
abling) state, where pages with old data may or may not have a checksum, yet. So I think it was an argument against staying in that state any longer than necessary. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Markus Wanner
cause that bitmap may well become a bottleneck itself. Plus there's the problem of making sure those pages are safe against corruptions, so you'd need to checksum the checksum bitmap... doesn't sound like a nice solution to me. This has certainly been discussed before. Regards Marku

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Markus Wanner
own to both > have a valid checksum on it and have its checksum bits set, *then* any > page that doesn't have both set bits and a matching checksum is garbage. >From that point in time on, we'd theoretically better use that bit as an additional checksum bit rather than requirin

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Markus Wanner
, I'd argue that prolonging the migration to spread the load would allow even big shops to go through this without much of an impact on performance. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-09 Thread Markus Wanner
ds to a prolonged migration and with time, the remaining VACUUM step becomes less and less frightening. Do you see any real foot-guns or other show-stoppers for permanently allowing that in-between-state? Or do we have other viable options that prolong the migration and thus spread the load better

Re: [HACKERS] Enabling Checksums

2012-11-09 Thread Markus Wanner
ge (provided you haven't ever turned checksumming on before). Maybe you want to save that step and still get the additional safety for newly dirtied pages, right? A use case worth supporting? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Enabling Checksums

2012-11-09 Thread Markus Wanner
d you never know what the committee is coming up with next. Apart from that, I'd like something more descriptive that just "checksums". Block checksums? Heap checksums? Data checksums? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical to physical page mapping

2012-10-30 Thread Markus Wanner
On 10/29/2012 12:05 PM, Robert Haas wrote: > OK, I'll stop babbling now... Not perceived as babbling here. Thanks for that nice round-up of options and ideas around the torn page problem. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] Global Sequences

2012-10-17 Thread Markus Wanner
and re-set an initial value, as a starting point for the next bulk of numbers that nextval() will return. currval() doesn't need to be changed or "hooked" at all, because it's a read-only operation. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Global Sequences

2012-10-17 Thread Markus Wanner
buted database as a transparent equivalent of a single-node system, I'd say the SQL standard applies to the entire distributed system. From that point of view, I'd rather argue that any "local-only" behavior violates the standard. Regards Markus Wanner -- Sent via pgsql-h

Re: [HACKERS] Global Sequences

2012-10-17 Thread Markus Wanner
r benefit). > which means its not going to be possible to provide > such a thing initially without slowing it down to the point we don't > actually get it at all. Sorry, I don't quite understand what you are trying to say, here. Overall, thanks for bringing this up. I'm g

Re: [HACKERS] Global Sequences

2012-10-16 Thread Markus Wanner
eally need an API that allows for implementations of options 1) and 2)? What I'd appreciate more is a common implementation for option 3) with an API to plug in different solutions to the underlying consensus problem. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] WAL_DEBUG logs spurious data

2012-10-11 Thread Markus Wanner
On 10/11/2012 04:06 PM, Tom Lane wrote: > Yeah, if we decide to stick with the limitation, some documentation > would be called for. I remember having run into this and having removed > functionality from an rm_desc function rather than question the premise. > But maybe the extra functionality is

Re: [HACKERS] WAL_DEBUG logs spurious data

2012-10-11 Thread Markus Wanner
Tom, On 10/11/2012 03:11 PM, Tom Lane wrote: > The original design intention was that rm_desc should not attempt to > print any such data, but obviously some folks didn't get the word. FWIW: in case the source code contains comments explaining that intention, I certainly missed them so far. Rega

[HACKERS] WAL_DEBUG logs spurious data

2012-10-11 Thread Markus Wanner
from WAL, the record is logged correctly: LOG: REDO @ 0/16F3270; LSN 0/16F329C: prev 0/16F3234; xid 688; len 16: Transaction - commit: 2012-10-11 09:31:17.790368-07; subxacts: 689 Attached is a possible fix. Regards Markus Wanner *** a/src/backend/access/transam/xlog.c --- b/src/backend/access

Re: [HACKERS] BlockNumber initialized to InvalidBuffer?

2012-07-13 Thread Markus Wanner
On 07/13/2012 11:33 AM, Markus Wanner wrote: > As another minor improvement, it doesn't seem necessary to repeatedly > set the rootBlkno. Sorry, my mail program delivered an older version of the patch, which didn't reflect that change. Here's what I intended to send. R

Re: [HACKERS] BlockNumber initialized to InvalidBuffer?

2012-07-13 Thread Markus Wanner
nor its contract. The only correct exit points are the "return"s > in the middle. I came to the same conclusion, yes. Looks like the additional asserts in the attached patch all hold true. As another minor improvement, it doesn't seem necessary to repeatedly set the rootBl

[HACKERS] BlockNumber initialized to InvalidBuffer?

2012-07-10 Thread Markus Wanner
tion gets overridden immediately, in which case it certainly doesn't matter. I don't know nor did I check whether or not it can ever be NULL. So this might not be a real issue at all. Regards Markus Wanner # InvalidBlockNumber is -1 (or rather 0x), while # the currently used

Re: [HACKERS] Pruning the TODO list

2012-06-30 Thread Markus Wanner
hould be named TODISCUSS, IDEAS, or something. But the current file name implies consensus. Wouldn't renaming solve that kind of misunderstanding? (..in the vain of "address(ing) real problems in the simplest way"..) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
ath we already have: subxids.overflowed. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
buffer that it will have wrapped around and clobbered the portion > the reader was interested in. That needs to be rare. Yeah. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
to be worth testing. If the whole thing > crashes and burns I hope I'll at least learn enough along the way to > design something better... That's always a good motivation. In that sense: happy hacking! Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] cheaper snapshots redux

2011-08-24 Thread Markus Wanner
Robert, On 08/25/2011 04:59 AM, Robert Haas wrote: > True; although there are some other complications. With a > sufficiently sophisticated allocator you can avoid mutex contention > when allocating chunks, but then you have to store a pointer to the > chunk somewhere or other, and that then requ

Re: [HACKERS] cheaper snapshots redux

2011-08-24 Thread Markus Wanner
case an imessage was left there for a longer amount of time, it blocked creation of new imessages, because the ring-buffer cycled around once and its head arrived back at the unconsumed imessage. IIUC (which might not be the case) the same issue applies for snapshots. Regards Markus Wanner

Re: [HACKERS] cheaper snapshots redux

2011-08-24 Thread Markus Wanner
Hello Dimitri, On 08/23/2011 06:39 PM, Dimitri Fontaine wrote: > I'm far from familiar with the detailed concepts here, but allow me to > comment. I have two open questions: > > - is it possible to use a distributed algorithm to produce XIDs, >something like Vector Clocks? > >Then each

Re: [HACKERS] POSIX question

2011-06-26 Thread Markus Wanner
cations, all trying to optimize for concurrency. The programmer of a multi-threaded application doesn't need to care much about concurrent allocations. He can allocate (and free) quite a lot of tiny chunks concurrently from shared memory. Regards Markus Wanner -- Sent via pgsql-hackers

Re: [HACKERS] Exporting closePGconn from libpq

2011-05-16 Thread Markus Wanner
t is relatively harmless. It's harmless, but I think it's also useless. Or why do you want to keep that (libpq-private) bit of memory around beyond PQfinish()? I'm not sure what language or VM you have in mind, but your description sounds like you are writing a wrapper by definition

Re: [HACKERS] "stored procedures" - use cases?

2011-05-12 Thread Markus Wanner
Hi, On 05/10/2011 02:55 PM, Robert Haas wrote: > On Mon, May 9, 2011 at 11:58 PM, Pavel Stehule > wrote: >> no - you are little bit confused :). CALL and function execution >> shares nothing. There is significant differences between function and >> procedure. Function is called only from executo

Re: [HACKERS] "full_page_writes" makes no difference?

2011-05-04 Thread Markus Wanner
ze) XLOG_BLCKSZ; > nbytes = npages * (Size) XLOG_BLCKSZ; > if (write(openLogFile, from, nbytes) != nbytes) > { > ... > } > > So, "nbytes" should always be multiples of XLOG_BLCKSZ, which in the > default case, is 8192. That observation seems correct. Regards Markus Wan

Re: [HACKERS] Proposal - asynchronous functions

2011-04-27 Thread Markus Wanner
On 04/26/2011 11:17 PM, Robert Haas wrote: > IIRC, we kind of got stuck on the prerequisite wamalloc patch, and that sunk > the whole thing. :-( Right, that prerequisite was the largest stumbling block. As I certainly mentioned back then, it should be possible to get rid of the imessages depend

Re: [HACKERS] Proposal - asynchronous functions

2011-04-26 Thread Markus Wanner
Robert, On 04/26/2011 02:25 PM, Robert Haas wrote: > We've talked about a number of features that could benefit from some > kind of "worker process" facility (e.g. logical replication, parallel > query). So far no one has stepped forward to build such a facility, > and I think without that this c

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-23 Thread Markus Wanner
On 03/23/2011 12:52 PM, Robert Haas wrote: > Yes. What this won't do is let you build a big load-balancing network > (at least not without great caution about what you assume). This sounds too strong to me. Session-aware load balancing is pretty common these days. It's the default mode of PgBou

  1   2   3   4   5   6   >