[HACKERS] Kernel changes to O_SYNC / O_DSYNC

[Craig Ringer]

Hi folks

It looks like there are some upcoming changes to the Linux kernel's 
handling of O_SYNC and O_DSYNC that I thought might be worth pointing out:


  http://lwn.net/Articles/350219/

--
Craig Ringer

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Largeobject Access Controls (r2460)

[Takahiro Itagaki]

Robert Haas  wrote:

> 2009/12/16 KaiGai Kohei :
>  ? ?long desc: When turned on, privilege checks on large objects perform 
>  with
>  ? ? ? ? ? ? ? backward compatibility as 8.4.x or earlier releases.

> Mostly English quality, but there are some other issues too.  Proposed
> patch attached.

I remember we had discussions about the version number, like
"Don't use '8.5' because it might be released as '9.0'", no?

Another comment is I'd like to keep 
for the first pg_largeobject in each topic.

Regards,
---
Takahiro Itagaki
NTT Open Source Software Center



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Largeobject Access Controls (r2460)

[KaiGai Kohei]

(2009/12/17 13:20), Robert Haas wrote:
> 2009/12/16 KaiGai Kohei:
>> (2009/12/17 7:25), Robert Haas wrote:
>>> On Thu, Dec 10, 2009 at 10:41 PM, Takahiro Itagaki
>>> wrote:

 KaiGai Koheiwrote:

> What's your opinion about:
> long desc: When turned on, privilege checks on large objects perform 
> with
>backward compatibility as 8.4.x or earlier releases.

 I updated the description as your suggest.

 Applied with minor editorialization,
 mainly around tab-completion support in psql.
>>>
>>> The documentation in this patch needs work.
>>
>> Are you talking about English quality? or Am I missing something to be
>> documented?
> 
> Mostly English quality, but there are some other issues too.  Proposed
> patch attached.

I have nothing to comment on English quality

Thanks for your fixups.
-- 
OSS Platform Development Division, NEC
KaiGai Kohei 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[tomas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Wed, Dec 16, 2009 at 04:45:54PM -0300, Alvaro Herrera wrote:
> to...@tuxteam.de wrote:
> 
> > (and as Andrew Dunstan pointed out off-list: I was wrong with my bold
> > assertion that one can squeeze infinitely many (arbitrary length)
> > strings between two given. This is not always the case).
> 
> Of course you can do that if you assume lexicographical order, or any
> other arbitrary order.

Hm. On strings (lexicographical order) there's always a "next" (using
the convention 'a' is the first symbol, 'z' the last and . the append
operator):

  next(X) == X . 'a'

(there is no string between X and X . 'a', and X < X . 'a')

But there generally no prev! (unless the last "character" of a string is
precisely 'a')

  prev("bob") == "baz.."

...and we just left the realm of finte-length strings. Strange structure
this space, has.

> some ordering on which this does not happen.  And in fact there is:
> order strings by length first, and then lexicographically.  If you do
> this then you have next() and prev() for any given string.  If you use
> ASCII only, you have a.next = b, and so on.

Yes, this is similar to the trick of ordering the rationals in a "snake
diagonal" like in [1], to prove their countability. But this sorting
isn't very useful (except in this clever proof).

> There is the argument that some languages do not sort lexicographically
> but this is also besides the point [...]

Totally agree.

> Defining strings with this ordering means you can have some useful
> ranges like [a-z], but then you cannot meaningfully use ranges for
> things like [aardvark - zulu] --- note that in this particular example,
> the order is reversed, because zulu comes before aardvark which is
> probably not what you want.   zzz.next = 

Yep. Its just useful to prove that the set of strings is countable (I
wonder whether our world would be different if we had chosen the
convention of looking up words in this "length+lexicographical" order in
the first place -- it reminds me of how Chinese-speaking people look up
symbols in a dictionary: number of strokes first).

> In short, I think that while it is possible to define ranges of strings,
> it is not as useful as one would like.

Unless you are willing to treat strings as non-discrete, somehow.

Regards
- 
[1] 

- -- tomás
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFLKbiFBcgs9XrR2kYRAnUYAJ0QBbWzNITA1KARLKdPTshFBSp/ZwCeIbin
Lwc/pRBYgKoaccGpn0beyu4=
=nViD
-END PGP SIGNATURE-

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Largeobject Access Controls (r2460)

[Robert Haas]

2009/12/16 KaiGai Kohei :
> (2009/12/17 7:25), Robert Haas wrote:
>> On Thu, Dec 10, 2009 at 10:41 PM, Takahiro Itagaki
>>   wrote:
>>>
>>> KaiGai Kohei  wrote:
>>>
 What's your opinion about:
    long desc: When turned on, privilege checks on large objects perform 
 with
               backward compatibility as 8.4.x or earlier releases.
>>>
>>> I updated the description as your suggest.
>>>
>>> Applied with minor editorialization,
>>> mainly around tab-completion support in psql.
>>
>> The documentation in this patch needs work.
>
> Are you talking about English quality? or Am I missing something to be
> documented?

Mostly English quality, but there are some other issues too.  Proposed
patch attached.

...Robert
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index fdff8b8..482aeac 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3125,9 +3125,8 @@
 
   
The catalog pg_largeobject holds the data making up
-   large objects.  A large object is identified by an OID of
-   pg_largeobject_metadata
-   catalog, assigned when it is created.  Each large object is broken into
+   large objects.  A large object is identified by an OID
+   assigned when it is created.  Each large object is broken into
segments or pages small enough to be conveniently stored as rows
in pg_largeobject.
The amount of data per page is defined to be LOBLKSIZE (which is currently
@@ -3135,10 +3134,12 @@
   
 
   
-   pg_largeobject should not be readable by the
-   public, since the catalog contains data in large objects of all users.
-   pg_largeobject_metadata is a publicly readable catalog
-   that only contains identifiers of large objects.
+   Prior to PostgreSQL 8.5, there was no permission structure
+   associated with large objects.  As a result,
+   pg_largeobject was publicly readable and could be
+   used to obtain the OIDs (and contents) of all large objects in the system.
+   This is no longer the case; use pg_largeobject_metadata to
+   obtain a list of large object OIDs.
   
 
   
@@ -3202,9 +3203,8 @@
   
 
   
-   The purpose of pg_largeobject_metadata is to
-   hold metadata of large objects, such as OID of its owner,
-   access permissions and OID of the large object itself.
+   The catalog pg_largeobject_metadata
+   holds metadata associated with large objects.
   
 
   
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 36d3a22..5e4b44a 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4825,22 +4825,19 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
   
   

-This allows us to tuen on/off database privilege checks on large
-objects. In the 8.4.x series and earlier release do not have
-privilege checks on large object in most cases.
-
-So, turning the lo_compat_privileges off means
-the large object feature performs in compatible mode.
+In PostgreSQL releases prior to 8.5, large objects
+did not have access privileges and were, in effect, readable and
+writable by all users.  Setting this variable to on
+disables the new privilege checks, for compatibility with prior
+releases.  The default is off.


-Please note that it is not equivalent to disable all the security
-checks corresponding to large objects.
-For example, the lo_import() and
+Setting this variable does not disable all security checks for
+large objects - only those for which the default behavior has changed
+in PostgreSQL 8.5.
+For example, lo_import() and
 lo_export() need superuser privileges independent
-from this setting as prior versions were doing.
-   
-   
-It is off by default.
+of this setting.

   
  
diff --git a/doc/src/sgml/lobj.sgml b/doc/src/sgml/lobj.sgml
index e5a680a..de246b0 100644
--- a/doc/src/sgml/lobj.sgml
+++ b/doc/src/sgml/lobj.sgml
@@ -59,6 +59,21 @@
 searches for the correct chunk number when doing random
 access reads and writes.

+
+   
+As of PostgreSQL 8.5, large objects have an owner
+and a set of access permissions, which can be managed using
+ and
+.  
+For compatibility with prior releases, see
+.
+SELECT privileges are required to read a large
+object, and 
+UPDATE privileges are required to write to or
+truncate it.
+Only the large object owner (or the database superuser) can unlink, comment
+on, or change the owner of a large object.
+   
   
 
   
@@ -438,60 +453,9 @@ SELECT lo_export(image.raster, '/tmp/motd') FROM image
 owning user.  Therefore, their use is restricted to superusers.  In
 contrast, the client-side import and export functions read and write files
 in the client's file system, using the permissions of the client program.
-The client-side functions can be 

Re: [HACKERS] SE-PostgreSQL/Lite Review

[KaiGai Kohei]

(2009/12/12 6:27), Robert Treat wrote:
>> One point. I'd like to introduce a use case without row-level granularity.
>>
>> The page.24 in this slide:
>>http://sepgsql.googlecode.com/files/JLS2009-KaiGai-LAPP_SELinux.pdf
>>
>> shows SELinux performs as a logical wall between virtual domains in
>> web-services. Unlike physical database separation, it also allows to
>> share a part of files/tables from multiple virtual hosts, because of
>> its flexibility.
>>
> 
> I got the impression that this is doable with current SEPostgres stuff, would
> be nice to see a little more detailed writeup on how to do it. Especially if
> it could be linked to the hosting providors page in the wiki.

Sorry, I missed to reply your message.

It needs to set up apache and selinux support module (mod_selinux.so)
correctly. This wiki article introduce the way to set up per virtualhost
separation using SELinux.

http://code.google.com/p/sepgsql/wiki/Apache_SELinux_plus?wl=en#Per_virtual-host_separation

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Does "verbose" Need to be Reserved?

[Andrew Gierth]

> "Euler" == Euler Taveira de Oliveira  writes:

 > David E. Wheeler escreveu:
 >> A list of such words, to be included in the Changes file, would be
 >> most useful I expect.

I'm guessing "Keywords listed as reserved in appendix C no longer work as
variable names in pl/pgsql, even though they might have worked in previous
versions" just about covers it.

 Euler> As we're on this topic, could we expose keywords in a system
 Euler> view like say pg_keywords?

select * from pg_get_keywords();

-- 
Andrew (irc:RhodiumToad)

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Does "verbose" Need to be Reserved?

[Euler Taveira de Oliveira]

David E. Wheeler escreveu:
> A list of such words, to be included in the Changes file, would be most 
> useful I expect.
> 
As we're on this topic, could we expose keywords in a system view like say
pg_keywords?


-- 
  Euler Taveira de Oliveira
  http://www.timbira.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] determine snapshot after obtaining locks for first statement

[Tom Lane]

"Kevin Grittner"  writes:
> Tom Lane  wrote:
>> (Besides which the lock acquired by UPDATE isn't exclusive and
>> wouldn't block anyway...)
 
> It blocks other UPDATEs.

Not at the table level.  If you could lock only at the tuple level
maybe you'd have something, but it seems like you can't find the
target tuples without having acquired a snapshot.

>> If he's talking about automatically taking an exclusive lock, I
>> doubt very many of our users would find that an improvement.
 
> I don't believe he's talking about a lock which excludes SELECTs on
> the data.

Well, you could take such a lock (one that blocks other UPDATEs but
not SELECTs) but it would be a clear loss of concurrency compared to
what we have now.  Unless I misunderstand what you're talking about,
it'd serialize all updates on a given table whether they conflict or
not.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: Add hstore_to_json()

[Robert Haas]

On Wed, Dec 16, 2009 at 5:58 PM, David E. Wheeler  wrote:
> On Dec 16, 2009, at 2:45 PM, Robert Haas wrote:
>
>> I like it.  The regression tests you've added seem to cover a lot of
>> cases that aren't really different without covering some that are
>> probably worth trying, like multiple key/value pairs.  Also, the
>> comment in the function you've added looks like a cut-and-paste from
>> somewhere else, which might not be the best way to document.  With
>> regard to the underlying issue, why can't we just use a StringInfo and
>> forget about it?
>
> Dunno. I just duped hstore_out(). I agree there should be more edge cases.
>
>> Also, your indentation is not entirely consistent.  If this gets
>> consensus, that will have to be fixed before it can be committed, so
>> it would be nice if you could do that rather than leaving it for the
>> eventual committer.
>
> The indentation is also largely copied; wouldn't pg_indent fix it?

Yeah, eventually, but that's not really a great way of dealing with it.

http://archives.postgresql.org/pgsql-hackers/2009-12/msg01208.php

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] determine snapshot after obtaining locks for first statement

[Kevin Grittner]

Tom Lane  wrote:
 
> With the code you give, the snapshot is acquired at the beginning
> of processing the UPDATE command, before it finds out that the
> target is t2a and acquires a lock on it.
 
Right.  The optimization would be to wait to determine the snapshot
for the UPDATE statement until we acquire the lock.  (This trick
only worked for the first statement in a database transaction.) 
Perhaps that's not feasible in PostgreSQL.  That's what I'm asking.
 
> (Besides which the lock acquired by UPDATE isn't exclusive and
> wouldn't block anyway...)
 
It blocks other UPDATEs.
 
>> The optimization Cahill describes is that for the first statement
>> in a transaction, the lock for the UPDATE is acquired before
>> obtaining the snapshot, so T2 succeeds after T1 commits.
> 
> If he's talking about automatically taking an exclusive lock, I
> doubt very many of our users would find that an improvement.
 
I don't believe he's talking about a lock which excludes SELECTs on
the data.  He's talking about reducing transaction aborts based on
the "First Committer Wins" rule.  Apparently it helped his
high-contention benchmarks considerably in all three isolation
levels he was testing -- snapshot, serializable snapshot, and
serializable S2PL.
 
If there's something fundamentally different about how PostgreSQL
does things, such that we *can't* get the lock before the snapshot,
that's fine -- I just thought maybe this was a trick we could use.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Largeobject Access Controls (r2460)

[KaiGai Kohei]

(2009/12/17 7:25), Robert Haas wrote:
> On Thu, Dec 10, 2009 at 10:41 PM, Takahiro Itagaki
>   wrote:
>>
>> KaiGai Kohei  wrote:
>>
>>> What's your opinion about:
>>>long desc: When turned on, privilege checks on large objects perform with
>>>   backward compatibility as 8.4.x or earlier releases.
>>
>> I updated the description as your suggest.
>>
>> Applied with minor editorialization,
>> mainly around tab-completion support in psql.
> 
> The documentation in this patch needs work.

Are you talking about English quality? or Am I missing something to be
documented?

-- 
OSS Platform Development Division, NEC
KaiGai Kohei 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] determine snapshot after obtaining locks for first statement

[Tom Lane]

"Kevin Grittner"  writes:
> Tom Lane  wrote:
>> Yes --- it's not an "optimization", it's necessary for basic
>> functionality to work correctly.
 
> Hmmm...  Testing seems to indicate that this doesn't work per the
> described optimization:

You'd need an explicit LOCK TABLE t2a after starting the transaction.
With the code you give, the snapshot is acquired at the beginning of
processing the UPDATE command, before it finds out that the target
is t2a and acquires a lock on it.  (Besides which the lock acquired
by UPDATE isn't exclusive and wouldn't block anyway...)
 
> The optimization Cahill describes is that for the first statement in
> a transaction, the lock for the UPDATE is acquired before obtaining
> the snapshot, so T2 succeeds after T1 commits.

If he's talking about automatically taking an exclusive lock, I doubt
very many of our users would find that an improvement.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] determine snapshot after obtaining locks for first statement

[Kevin Grittner]

Tom Lane  wrote:
> "Kevin Grittner"  writes:
>> The Cahill thesis mentions an interesting optimization -- they
>> defer determination of the snapshot until after any locks
>> required for the first statement have been acquired.  Where the
>> first statement was, for example, an UPDATE, this reduced
>> re-reads or rollbacks in the face of concurrent modifications.
>  
>> Does PostgreSQL currently do this?
> 
> Yes --- it's not an "optimization", it's necessary for basic
> functionality to work correctly.
 
Hmmm...  Testing seems to indicate that this doesn't work per the
described optimization:
 
T1: start transaction isolation level serializable;
START TRANSACTION
T2: start transaction isolation level serializable;
START TRANSACTION
T1: update t2a set c2 = c2 - 1 where c1 = 1;
UPDATE 1
T2: update t2a set c2 = c2 - 1 where c1 = 1;
[blocks]
T1: commit;
COMMIT
T2: [unblocks]
ERROR:  could not serialize access due to concurrent update
 
The optimization Cahill describes is that for the first statement in
a transaction, the lock for the UPDATE is acquired before obtaining
the snapshot, so T2 succeeds after T1 commits.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: Add hstore_to_json()

[David E. Wheeler]

On Dec 16, 2009, at 2:45 PM, Robert Haas wrote:

> I like it.  The regression tests you've added seem to cover a lot of
> cases that aren't really different without covering some that are
> probably worth trying, like multiple key/value pairs.  Also, the
> comment in the function you've added looks like a cut-and-paste from
> somewhere else, which might not be the best way to document.  With
> regard to the underlying issue, why can't we just use a StringInfo and
> forget about it?

Dunno. I just duped hstore_out(). I agree there should be more edge cases.

> Also, your indentation is not entirely consistent.  If this gets
> consensus, that will have to be fixed before it can be committed, so
> it would be nice if you could do that rather than leaving it for the
> eventual committer.

The indentation is also largely copied; wouldn't pg_indent fix it?

Best,

David

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Does "verbose" Need to be Reserved?

[David E. Wheeler]

On Dec 16, 2009, at 2:29 PM, Tom Lane wrote:

>> Looking at it more closely, this is likely to be fallout from the
>> plpgsql lexer/parser changes; it probably worked before only because
>> plpgsql was doing its own thing rather than using the main lexer.
> 
> Hmm .. yeah, words that are reserved according to the main grammar
> could have worked as plpgsql variable names before, since they always
> got replaced with "$n" before the main grammar saw them.

A list of such words, to be included in the Changes file, would be most useful 
I expect.

Best,

David
-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] determine snapshot after obtaining locks for first statement

[Tom Lane]

"Kevin Grittner"  writes:
> The Cahill thesis mentions an interesting optimization -- they defer
> determination of the snapshot until after any locks required for the
> first statement have been acquired.  Where the first statement was,
> for example, an UPDATE, this reduced re-reads or rollbacks in the
> face of concurrent modifications.
 
> Does PostgreSQL currently do this?

Yes --- it's not an "optimization", it's necessary for basic
functionality to work correctly.  See for example the last para at
http://www.postgresql.org/docs/8.4/static/applevel-consistency.html

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: Add hstore_to_json()

[Robert Haas]

On Wed, Dec 16, 2009 at 2:28 PM, David E. Wheeler  wrote:
> I just realized that this was easy to do, and despite my complete lack of C 
> skillz was able to throw this together in a couple of hours. It might be 
> handy to some, though the possible downsides are:
>
> * No json_to_hstore().
> * Leads to requests for hstore_to_yaml(), hstore_to_xml(), etc.
> * Andrew Gierth said “no” when I suggested it.
>
> But it's kind of handy, too. Thoughts?

I like it.  The regression tests you've added seem to cover a lot of
cases that aren't really different without covering some that are
probably worth trying, like multiple key/value pairs.  Also, the
comment in the function you've added looks like a cut-and-paste from
somewhere else, which might not be the best way to document.  With
regard to the underlying issue, why can't we just use a StringInfo and
forget about it?

Also, your indentation is not entirely consistent.  If this gets
consensus, that will have to be fixed before it can be committed, so
it would be nice if you could do that rather than leaving it for the
eventual committer.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Jeff Davis]

On Wed, 2009-12-16 at 14:29 +0100, to...@tuxteam.de wrote:
> This alone would practically preclude discrete -- int and float would
> behave quite differently (float's "granules" growing towards the edges
> or having to choose a bigger granule for float than for int in the first
> place).

It may be an argument for a different range type name, or trying to spot
obviously dangerous things and throw an error.

But I don't think it's an argument to prevent a superuser from defining
a discrete range of whatever he or she wants, as long as they provide
the necessary functions.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Does "verbose" Need to be Reserved?

[Tom Lane]

Andrew Gierth  writes:
> Looking at it more closely, this is likely to be fallout from the
> plpgsql lexer/parser changes; it probably worked before only because
> plpgsql was doing its own thing rather than using the main lexer.

Hmm .. yeah, words that are reserved according to the main grammar
could have worked as plpgsql variable names before, since they always
got replaced with "$n" before the main grammar saw them.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Largeobject Access Controls (r2460)

[Robert Haas]

On Thu, Dec 10, 2009 at 10:41 PM, Takahiro Itagaki
 wrote:
>
> KaiGai Kohei  wrote:
>
>> What's your opinion about:
>>   long desc: When turned on, privilege checks on large objects perform with
>>              backward compatibility as 8.4.x or earlier releases.
>
> I updated the description as your suggest.
>
> Applied with minor editorialization,
> mainly around tab-completion support in psql.

The documentation in this patch needs work.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Jeff Davis]

On Wed, 2009-12-16 at 13:59 -0500, Tom Lane wrote:
> For example, if you're trying to do classroom scheduling, it might be
> useful to constrain the periods to start and end on hour boundaries
> --- but the next thing you'll want is to have it know that the "next"
> slot after 5pm Friday is 8am Monday.  Except on holidays.  And then
> there's the fact that my alma mater starts most hour-long classes on
> the half hour.

Data types are only a first-level constraint -- a domain of reasonable
values. The class isn't going to start on a fraction-of-a-minute
boundary, so it would be reasonable to reject those values early.

I never suggested that next() should be such a highly business-dependent
function as you suggest above (skipping holidays, etc); it should just
return the next value in the domain (if it's discrete).

Surely you wouldn't suggest that the ipv4 data type's next() function
should skip over addresses that aren't in a valid subnet on your
network. But you seem to think those make useful discrete ranges.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Largeobject Access Controls and pg_migrator

[Bruce Momjian]

Takahiro Itagaki wrote:
> 
> KaiGai Kohei  wrote:
> 
> > Can SELECT lo_create(16385); help this situation?
> 
> SELECT lo_create(loid) FROM (SELECT DISTINCT loid FROM pg_largeobject) AS t
> 
> would work for pg_migrator.
> 
> > I'm not clear whether we also check pg_largeobejct has chunks with same
> > LOID on lo_create(). In the regular operation, it shall never happen.
> 
> I think the omission is a reasonable optimization.

Thanks, I have updated pg_migrator to use your suggested method.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] determine snapshot after obtaining locks for first statement

[Kevin Grittner]

The Cahill thesis mentions an interesting optimization -- they defer
determination of the snapshot until after any locks required for the
first statement have been acquired.  Where the first statement was,
for example, an UPDATE, this reduced re-reads or rollbacks in the
face of concurrent modifications.
 
Does PostgreSQL currently do this?  If not, would it make sense?
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Scott Bailey]

Tom Lane wrote:

Martijn van Oosterhout  writes:

However, it does seem reasonable to allow people to restrict, either by
typmod or a check constraint the kinds of values that can be stored in
a particular column. Then an application can decide which way they want
their intervals to work and have the database enforce it.


Sure --- the range datatype should absolutely provide inquiry functions
that let you determine all the properties of a range, so something like
"CHECK (is_open_on_right(col))" would work for that.  I'm of the opinion
that we must not usurp typmod for range behavior --- the right thing is
to pass that through to the contained type, just as we do with arrays.

(Note that a range over timestamp(0) would eliminate at least some of
the platform dependencies we've been arguing about.  I'm still quite
dubious that "next timestamp" is anything except evidence that you've
misformulated your problem, though.)

regards, tom lane


Well our work is based on over 15 years of temporal research (not by us) 
and numerous books from Snodgrass, Date and Celko; as well as partial 
implementations in other databases. So its not like we took a blue pill 
this weekend and woke up with this hair-brained idea.


I understand your concern. But I think the objections are based more on 
implementation details with float timestamp rather than conceptually.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Jeff Davis]

On Wed, 2009-12-16 at 15:57 -0500, Tom Lane wrote:
> Making it explicit doesn't fix the fact that you can't rely on the
> arithmetic to be exact.

Can't rely on what arithmetic to be exact? Int64 timestamps should
clearly work for granules of 1 second.

If the administrator can choose a timestamp format that can't accurately
represent whole seconds, that's not an argument that modeling based on
whole seconds is worthless. We can restrict float timestamps for ranges
as a special case, if we're that worried about people misusing them.

> > I still have not seen an answer to the problem of changing the
> > representation of a continuous range. If you have the continuous range
> > [5, 10], you're pretty much stuck with that representation, even if the
> > application is expecting things in the form [ ).
> 
> That is not our problem.  It's the application's problem if it can't
> handle the concept.  You might as well be complaining that type numeric
> is broken because it can represent values that will fail to fit into
> float8 when some application tries to force them into that form.

...except that we support float8. So if applications like to work float8
float8, we let them.

You're arguing that we should not support discrete time ranges (or even
allow such a type to be defined by a superuser) even though applications
and users may happen to model time that way.

Who are we to argue that all of those people are so wrong that we won't
even support their type? Especially when they may have just finished a
couple books on the subject[1][2] which told them to model it that way?

> > And to further make the case for allowing user-defined discrete ranges,
> > what about ip4r?
> 
> What about it?  I don't have a problem with the concept that next() is
> well defined for some datatypes.

Ok, so we'll allow users to specify user-defined types for discrete
ranges? How should that be specified, and how will that differ from
earlier proposals?

Earlier you proposed that we hard-wire the set of types that are allowed
to be used for discrete ranges:

http://archives.postgresql.org/pgsql-hackers/2009-12/msg01278.php

Regards,
Jeff Davis

[1] "Temporal Data and the Relational Model" by C.J. Date, et al.
[2] "Developing Time-Oriented Database Applications in SQL" by Richard
Snodgrass.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Tom Lane]

Martijn van Oosterhout  writes:
> However, it does seem reasonable to allow people to restrict, either by
> typmod or a check constraint the kinds of values that can be stored in
> a particular column. Then an application can decide which way they want
> their intervals to work and have the database enforce it.

Sure --- the range datatype should absolutely provide inquiry functions
that let you determine all the properties of a range, so something like
"CHECK (is_open_on_right(col))" would work for that.  I'm of the opinion
that we must not usurp typmod for range behavior --- the right thing is
to pass that through to the contained type, just as we do with arrays.

(Note that a range over timestamp(0) would eliminate at least some of
the platform dependencies we've been arguing about.  I'm still quite
dubious that "next timestamp" is anything except evidence that you've
misformulated your problem, though.)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Jeff Davis]

On Wed, 2009-12-16 at 15:46 -0500, Tom Lane wrote:
> > Huh? We're miscommunicating somewhere.
> 
> Yeah, apparently.  By open-ended I meant -infinity left bound, or null
> left bound if you prefer.  Not sure if there's a better term.

But my proposal allowed both of those things with various flag settings
(because it has a flag byte in the latest proposal). I said so
explicitly.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Kevin Grittner]

Robert, Please forgive a couple editorial inserts to your statement
-- I hope it clarifies.  If I've distorted your meaning, feel free
to straighten me out.   :-)
 
Robert Haas  wrote:
 
> This thread veered off into a discussion of the traditional
> [predicate locking] technique, rather than the [serializable] one
> in the paper.  I think that's the part Alvaro was responding to.
 
If you're right about Alvaro's concern -- my rough understanding is
that HOT creates a linked lists of tuples which are mutations of one
another without altering any value which is part of an index.  If
that's anywhere near a correct understanding, I can't see how there
would be a problem with using the described locking techniques with
any tuple on the list.
 
As an aside, the thesis mentions smart use of multiple locking
granularities only under the "future work" section.  I can't see an
implementation being considered production quality without that, as
without it there would be no way to constrain the space required to
track the locks.  But there is no shortage of literature on how to
do that, so I view such discussions as more appropriate to low-level
implementation discussions should we ever get serious about using
the techniques which are the main thrust of Dr. Cahill's thesis.
 
If anyone is interested in reviewing recent literature on these
techniques, Dr. Cahill seemed to like (Hellerstein et al., 2007), to
the point where I may well track it down when I'm done pondering the
work which I referenced at the start of this thread.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Martijn van Oosterhout]

On Wed, Dec 16, 2009 at 03:57:44PM -0500, Tom Lane wrote:
> Jeff Davis  writes:
> > I still have not seen an answer to the problem of changing the
> > representation of a continuous range. If you have the continuous range
> > [5, 10], you're pretty much stuck with that representation, even if the
> > application is expecting things in the form [ ).
> 
> That is not our problem.  It's the application's problem if it can't
> handle the concept.  You might as well be complaining that type numeric
> is broken because it can represent values that will fail to fit into
> float8 when some application tries to force them into that form.

However, it does seem reasonable to allow people to restrict, either by
typmod or a check constraint the kinds of values that can be stored in
a particular column. Then an application can decide which way they want
their intervals to work and have the database enforce it.

(Intermediate values may become a different kind, just as long as the
result being stored it the right kind.)

Have a nice day,
-- 
Martijn van Oosterhout  http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while 
> boarding. Thank you for flying nlogn airlines.


signature.asc
Description: Digital signature

Re: [HACKERS] Range types

[Tom Lane]

Jeff Davis  writes:
> On Wed, 2009-12-16 at 13:59 -0500, Tom Lane wrote:
>> The argument for having
>> granularity wired into the datatype seems to boil down to just space
>> savings.  I don't find that compelling enough to justify code
>> contortions and user-visible restrictions on functionality.

> The argument (at least from me) is that discrete ranges have better
> semantics. The counterargument was that the granularity of a timestamp
> is an implementation detail. So I countered by making it explicit.

Making it explicit doesn't fix the fact that you can't rely on the
arithmetic to be exact.

> I still have not seen an answer to the problem of changing the
> representation of a continuous range. If you have the continuous range
> [5, 10], you're pretty much stuck with that representation, even if the
> application is expecting things in the form [ ).

That is not our problem.  It's the application's problem if it can't
handle the concept.  You might as well be complaining that type numeric
is broken because it can represent values that will fail to fit into
float8 when some application tries to force them into that form.

> And to further make the case for allowing user-defined discrete ranges,
> what about ip4r?

What about it?  I don't have a problem with the concept that next() is
well defined for some datatypes.  I just have a problem with the concept
that it's well-defined for timestamps.  It's not, and I don't believe
that forcing it to have some definition is useful in the real world
(which, as a rule, isn't going to fit the simplifying assumptions you
have to make to make it even sort-of work).

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Tom Lane]

Jeff Davis  writes:
> On Wed, 2009-12-16 at 12:50 -0500, Tom Lane wrote:
>> I'm still not exactly clear on what the use-case is for discrete
>> timestamp ranges, and I wonder how many people are going to be happy
>> with a representation that can't handle a range that's open-ended
>> on the left.

> Huh? We're miscommunicating somewhere.

Yeah, apparently.  By open-ended I meant -infinity left bound, or null
left bound if you prefer.  Not sure if there's a better term.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Tom Lane]

Martijn van Oosterhout  writes:
> But a period type will take just one or two more bytes if you don't
> require alignment. Alignment on a varlena type seems silly anyway,
> since you'll be aligning the header byte rather than the content.

You might still end up paying the alignment overhead after the field,
of course.  But avoiding embedding the alignment in the type itself
seems worth doing.

One idea that might be interesting to consider in this regard is
force-packing varlena range values.  Consider a text range('a', 'b').
The datums are likely to come in with 4-byte headers requiring
alignment.  If we have the smarts to force them to 1-byte header form
inside the varlena range value, not only do we save bytes right there,
but we don't have to expend cycles to copy them somewhere to re-align
them before we can pass them to the datatype-specific functions.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Alvaro Herrera]

Tom Lane wrote:
> Alvaro Herrera  writes:
> > In short, I think that while it is possible to define ranges of strings,
> > it is not as useful as one would like.
> 
> Note it is not the *range* that is the problem, it is the assumption
> that there's a unique "next" string.  There's no unique next in the
> reals or rationals either, but we have no problem considering intervals
> over those sets.

Yeah, agreed.  It's easy (I think) to define more useful ranges of
strings if you don't insist in having "next".

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Jeff Davis]

On Wed, 2009-12-16 at 13:59 -0500, Tom Lane wrote:
> The argument for having
> granularity wired into the datatype seems to boil down to just space
> savings.  I don't find that compelling enough to justify code
> contortions and user-visible restrictions on functionality.

The argument (at least from me) is that discrete ranges have better
semantics. The counterargument was that the granularity of a timestamp
is an implementation detail. So I countered by making it explicit.

Space savings is not crucial, but it would be frustrating to needlessly
waste space.

I still have not seen an answer to the problem of changing the
representation of a continuous range. If you have the continuous range
[5, 10], you're pretty much stuck with that representation, even if the
application is expecting things in the form [ ).

>From an application's standpoint, you probably want to get the
information about a range as separate columns (as opposed to parsing a
string). So an application expecting data in [) format might use a query
like:

 select ..., first(myperiod), next(myperiod) from mytable;

That gives the application complete information about the range. You can
even make a view over a table like that to make it even more transparent
to the application. It's not entirely unreasonable that many such
applications exist; there are many presentations and tutorials that have
been telling people to use a "start" and "end" column, and assume that
the start is inclusive and the end is exclusive.

If there is some other application that expects data in (] format, you
just use the query:

  select ..., prior(myperiod), last(myperiod) from mytable;

With discrete ranges, that all just works (barring overflow or special
values).

With continuous ranges, first() or next() might fail on some values that
were produced by some other application. Really, for continuous ranges,
you'd need to have a query like:

  select ..., start(myperiod), start_inclusive(myperiod),
end(myperiod), end_inclusive(myperiod) from mytable;

in order to have all of the information. And that means that the
application needs a full implementation of a range type to understand
the inclusivity and produce a correct result.

And to further make the case for allowing user-defined discrete ranges,
what about ip4r? That's a discrete range, and it's user-defined. And
that probably means other useful discrete ranges will be invented, as
well.

If we want to say that there is no discrete TIMESTAMP range by default,
and that the superuser has to define it, that's one thing. But if we say
that the only acceptable base types for discrete ranges will be
hard-wired, that's way too restrictive. If nothing else, it makes some
system types "special" which we have not done very many other places.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Does "verbose" Need to be Reserved?

[Tom Lane]

"David E. Wheeler"  writes:
> I asked on IRC, and Andrew “RhodiumToad” Gierth pointed out that
> it became a reserved word at some point.

"Some point" would have been around the time VACUUM VERBOSE got
invented, ie January 1997 according to the CVS logs.  We can't unreserve
it until you're ready to break that syntax (is VERBOSE a table name
or a flag for a database-wide vacuum?).  Don't hold your breath.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Robert Haas]

On Wed, Dec 16, 2009 at 1:30 PM, Kevin Grittner
 wrote:
> Alvaro Herrera  wrote:
>
>> So you'd have to disable HOT updates when true serializability was
>> active?
>
> I wouldn't think so; but someone familiar with HOT logic could
> probably determine whether the unmodified algorithm could be used by
> reviewing the "simplifying assumptions" near the bottom of page 42,
> and the page about "Generalizing to other database engines" (section
> 4.8).  I think those portions might stand on their own without
> reading the rest of the document.

This thread veered off into a discussion of the traditional technique,
rather than the one in the paper.  I think that's the part Alvaro was
responding to.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Tom Lane]

Alvaro Herrera  writes:
> In short, I think that while it is possible to define ranges of strings,
> it is not as useful as one would like.

Note it is not the *range* that is the problem, it is the assumption
that there's a unique "next" string.  There's no unique next in the
reals or rationals either, but we have no problem considering intervals
over those sets.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Robert Haas]

On Wed, Dec 16, 2009 at 1:29 PM, Boszormenyi Zoltan  wrote:
> Robert Haas írta:
>> On Wed, Dec 16, 2009 at 1:25 PM, Robert Haas  wrote:
>>
>>> On Wed, Dec 16, 2009 at 1:14 PM, Alvaro Herrera
>>>  wrote:
>>>
 Robert Haas escribió:

> On Wed, Dec 16, 2009 at 10:29 AM, Andres Freund  
> wrote:
>
>> On Wednesday 16 December 2009 16:24:42 Robert Haas wrote:
>>
   Inserts and deletes follow the same protocol, obtaining an exclusive
   lock on the row after the one being inserted or deleted. The result
   of this locking protocol is that a range scan prevents concurrent
   inserts or delete within the range of the scan, and vice versa.

 That sounds like it should actually work.

>>> Only if you can guarantee that the database will access the rows using
>>> some particular index.  If it gets to the data some other way it might
>>> accidentally circumvent the lock.  That's kind of a killer in terms of
>>> making this work for PostgreSQL.
>>>
>> Isnt the whole topic only relevant for writing access? There you have to
>> access the index anyway.
>>
> Yeah, I guess you have to insert the new tuple.  I guess while you
> were at it you might check whether the next tuple is locked...
>
 So you'd have to disable HOT updates when true serializability was
 active?

>>> I thought about that, but I don't think so.   HOT only applies to
>>> updates, and predicate locking only applies to inserts.  Unless I have
>>> my head in the sand?
>>>
>>
>> Err, no, wait.  Predicate locking can apply to updates, but since HOT
>> updates never update an indexed column, I think we might still be OK?
>>
>
> A predicate can include columns from an index plus others.
> Am I missing something?

Hmm, interesting point.  In that case you couldn't use the index to
enforce predicate locking under MVCC without disabling HOT.  But there
will be other cases where that wouldn't help anyway - a predicate
could also include unindexed columns exclusively.  For those, the
traditional approach (not the one discussed in this paper) probably
requires locking against any heap insert, or checking each new heap
insert against the constraint, or... something.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Martijn van Oosterhout]

On Wed, Dec 16, 2009 at 10:57:19AM -0800, Scott Bailey wrote:
> Ok, silly question here. But how do you determine the length of a  
> continuous range? By definition length of [a, b) and (a, b] = b-a. But  
> what about (a,b) and [a,b]? Are we saying that because they are  
> continuous, the difference between values included in the range and  
> those excluded are so infinitesimally small so as not to matter? Thus  
> length (a,b) == length [a,b] == length [a,b)? And if that is the case,  
> does the inclusiveness of the range really even matter?

Short answer: Yes

Longer answer: You need to decide on your definition of "length" and
what you usually use is the "measure". And yes, the difference between
the two is so called "measure 0" and thus has no effect on the length.

Note the measure has to be done considering the intervals as intervals
on a real line. The integers by themselves have no measure (they are
countable). So for the "length" of a set of integers you might consider
the count of the set.

http://planetmath.org/encyclopedia/ProofThatTheOuterLebesgueMeasureOfAnIntervalIsItsLength.html
http://en.wikipedia.org/wiki/Outer_measure

As for "continuous", as you use it above is not a way I recognise.
There are contiguous sets, but they are something else.

Have a nice day,
-- 
Martijn van Oosterhout  http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while 
> boarding. Thank you for flying nlogn airlines.


signature.asc
Description: Digital signature

Re: [HACKERS] Does "verbose" Need to be Reserved?

[Andrew Gierth]

> "David" == "David E Wheeler"  writes:

 David> Hey All,
 David> I was just getting a new version of pgTAP ready for release, and while 
testing it on HEAD, I got this error:

 David> + psql:pgtap.sql:5789: ERROR:  syntax error at end of input
 David> + LINE 28: IF verbose THEN RETURN NEXT diag(tests[i] 
||...
 David> +^

 David> I asked on IRC, and Andrew “RhodiumToad” Gierth pointed out
 David> that it became a reserved word at some point. I'm fine to
 David> rename my variable, but Andew and I were wondering if it's
 David> really necessary for "verbose" to be reserved, since it's not
 David> in the spec.

Looking at it more closely, this is likely to be fallout from the
plpgsql lexer/parser changes; it probably worked before only because
plpgsql was doing its own thing rather than using the main lexer.

VERBOSE has been reserved all along in order to distinguish 
'vacuum verbose;' from 'vacuum tablename;' and so on; but it's still an
interesting pitfall for plpgsql users.

-- 
Andrew (irc:RhodiumToad)

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Martijn van Oosterhout]

On Tue, Dec 15, 2009 at 04:29:26PM -0800, Jeff Davis wrote:
> On Tue, 2009-12-15 at 18:06 -0600, decibel wrote:
> > Now that varlena's don't have an enormous fixed overhead, perhaps it's
> > worth looking at using them. Obviously some operations would be
> > slower, but for your stated examples of auditing and history, I
> > suspect that you're not going to notice the overhead that much.
> 
> For most varvarlena types, you only get stuck with the full alignment
> burden if you get unlucky. In this case, we're moving from 16 bytes to
> 17, which really means 24 bytes with alignment. Try creating two tables:
> 
>   create table foo(i int8, t1 timestamp, t2 timestamp);
>   create table bar(i int8, c "char", t1 timestamp, t2 timestamp);

But a period type will take just one or two more bytes if you don't
require alignment. Alignment on a varlena type seems silly anyway,
since you'll be aligning the header byte rather than the content.

In the implementation you may need to copy the content before
processing to satisfy the alignment of the contained type, but that's
just a SMOP.

Have a nice day,
-- 
Martijn van Oosterhout  http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while 
> boarding. Thank you for flying nlogn airlines.


signature.asc
Description: Digital signature

[HACKERS] Does "verbose" Need to be Reserved?

[David E. Wheeler]

Hey All,

I was just getting a new version of pgTAP ready for release, and while testing 
it on HEAD, I got this error:

+ psql:pgtap.sql:5789: ERROR:  syntax error at end of input
+ LINE 28: IF verbose THEN RETURN NEXT diag(tests[i] ||...
+^

I asked on IRC, and Andrew “RhodiumToad” Gierth pointed out that it became a 
reserved word at some point. I'm fine to rename my variable, but Andew and I 
were wondering if it's really necessary for "verbose" to be reserved, since 
it's not in the spec.

Thanks,

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Alvaro Herrera]

to...@tuxteam.de wrote:

> (and as Andrew Dunstan pointed out off-list: I was wrong with my bold
> assertion that one can squeeze infinitely many (arbitrary length)
> strings between two given. This is not always the case).

Of course you can do that if you assume lexicographical order, or any
other arbitrary order.  The interesting point is whether there exists
some ordering on which this does not happen.  And in fact there is:
order strings by length first, and then lexicographically.  If you do
this then you have next() and prev() for any given string.  If you use
ASCII only, you have a.next = b, and so on.

There is the argument that some languages do not sort lexicographically
but this is also besides the point -- you only need to find *some* way
to sort the characters in the alphabet.  If you dictate that in your
ordering "á" comes before "à" and both after "a", and all of them before
b, then you know that a.next = á and á.next = à and à.next = b.  (Note
that I have also dictated that there is no other character that sorts
after a and before b, which is perfectly possible because the alphabet
is fixed for any given language.  You could use the complete list of
characters coded in a given set of Unicode planes, or even extant all
planes, to obtain the same result).

Defining strings with this ordering means you can have some useful
ranges like [a-z], but then you cannot meaningfully use ranges for
things like [aardvark - zulu] --- note that in this particular example,
the order is reversed, because zulu comes before aardvark which is
probably not what you want.   zzz.next = 

In short, I think that while it is possible to define ranges of strings,
it is not as useful as one would like.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: Spurious "22" in hstore.sgml

[Magnus Hagander]

On Wed, Dec 16, 2009 at 20:34, David E. Wheeler  wrote:
> *** a/doc/src/sgml/hstore.sgml
> --- b/doc/src/sgml/hstore.sgml

Heh, interesting. That clearly shouldn't be there. Applied.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Peter Eisentraut]

On ons, 2009-12-16 at 10:49 -0500, Tom Lane wrote:
> Marko Kreen  writes:
> > So the plain-C89 compilers would be downgraded to "second-class"
> > targets, not worth getting max performance out of them.
> 
> Hm?  Failing to inline is already a performance hit, which is why
> Kurt got interested in this in the first place.
> 
> I think you're way overthinking this.  Where we started was just
> a proposal to try to expand the set of inline-ing compilers beyond
> "gcc only".  I don't see why we need to do anything but that.  The
> code is fine as-is except for the control #ifdefs.

I think the ifdefs should just be HAVE_INLINE && !MSVC, right?


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] PATCH: Spurious "22" in hstore.sgml

[David E. Wheeler]

*** a/doc/src/sgml/hstore.sgml
--- b/doc/src/sgml/hstore.sgml
***
*** 278,284 
get hstore's keys as a set
skeys('a=>1,b=>2')

! 22
  a
  b
  
--- 278,284 
get hstore's keys as a set
skeys('a=>1,b=>2')

! 
  a
  b
  


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Jeff Davis]

On Wed, 2009-12-16 at 12:50 -0500, Tom Lane wrote:
> I'm still not exactly clear on what the use-case is for discrete
> timestamp ranges, and I wonder how many people are going to be happy
> with a representation that can't handle a range that's open-ended
> on the left.

Huh? We're miscommunicating somewhere. Discrete ranges are values, and
those values can be displayed a number of different ways. That's one of
the biggest advantages.

The very same discrete range can be displayed as open-open, closed-open,
open-closed, or closed-closed. There are edge cases, like how infinity
is never closed, and overflow conditions. But generally speaking, you
have more options for presenting a discrete range than a continuous
range.

The range [5, 7) is equal to the set {5, 6} and equal to the ranges:
(4,7), (4,6], [5,7), and [5,6]. One application can insert it as [5,7)
and another can read it as (4,6].

That's the use case: the application's preferences don't have to match.
It's OK to mix various representation preferences, because you can
convert between them. The on disk format happens to hint at one
particular canonical form, but doesn't enforce that on anyone.

> Huh?  You're not going to be able to have a special case data
> representation for one or two data types at the same time as you have a
> function-based datatype-independent concept of a parameterized range
> type.  Well, maybe you could have special code paths for just date and
> timestamp but it'd be horrid.

They aren't supposed to be exactly the same API, I said that from the
start. There are API differences between continuous and discrete ranges,
and we shouldn't ignore them.

One important differences is that (barring overflow conditions and
special values) prior, first, last, and next are defined for all
discrete range values, but not for all continuous range values.

For instance, the discrete range [5,7) has prior=4, first=5, last=6,
next=7.

Whereas the continuous range [5,7) has prior=undef, first=5, last=undef,
next=7.

We could define one API, that treats discrete and continuous ranges
differently. But you'll never be able to transform a continuous range to
a different representation, while you can do so with a discrete range.

> More importantly, the notion of a representation granule is still 100%
> wishful thinking for any inexact-representation datatype, which is going
> to be a severe crimp in getting this accepted for timestamp, let alone
> defining it in a way that would allow users to try to apply it to
> floats.  Float timestamps might not be the default case anymore but they
> are still supported.

If the only objection is that a superuser can confuse the system by
poorly defining a range type on a non-default build, I think that
objection can be overcome.

> I think you should let go of the feeling that you have to shave bytes
> off the storage format.  You're creating a whole lot of work for
> yourself and a whole lot of user-visible corner cases in return for
> what ultimately isn't much.

This isn't just to shave bytes. It's also because I like the semantics
of discrete ranges for many cases.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] pinging Dano

[Markus Wanner]

Hi,

who is the main editor named "Dano" of the wiki page about Parallel 
Query Execution 
(http://wiki.postgresql.org/wiki/Parallel_Query_Execution), please speak up.


Is there any code or patch available ATM? What "discussion with Tom and 
Simon" is that page referring to?


Regards

Markus Wanner


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] PATCH: Add hstore_to_json()

[David E. Wheeler]

I just realized that this was easy to do, and despite my complete lack of C 
skillz was able to throw this together in a couple of hours. It might be handy 
to some, though the possible downsides are:

* No json_to_hstore().
* Leads to requests for hstore_to_yaml(), hstore_to_xml(), etc.
* Andrew Gierth said “no” when I suggested it.

But it's kind of handy, too. Thoughts?

Best,

David




hstore_to_json.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Jeff Davis]

On Wed, 2009-12-16 at 12:42 -0500, Robert Haas wrote:
> On Wed, Dec 16, 2009 at 12:31 PM, Jeff Davis  wrote:
> > There's one problem, and that's for timestamptz ranges with intervals
> > that include days and months. Timezone adjustments are just not
> > well-defined for that kind of granule (nor would it be particularly
> > useful even if it magically worked), so this would have to be blocked
> > somehow. I think that's a special case, and we could provide the user
> > with a nice error message telling the user to use a date or timestamp
> > range instead.
> 
> This seems like a fairly special-purpose type.  You'd be targeting it
> at people who are very concerned with storing large numbers of these
> (so they really care about space consumption) but for some reason
> don't need to mix days and months (actually, the current interval
> representation stores days, months, and seconds separately).  I
> certainly think this might be useful to some people but it doesn't
> really sounds like a general range type facility, since it seems to
> involve some hacks that are fairly datatype-specific.

My statement should have read "days or months". In other words, you
can't have a timestamptz range with a granularity of '3 days'. But if
that's your granularity, as Scott says, you should be using a date
range, not a timestamptz range.

Timestamptz ranges are only really useful when you have a granularity
measured in seconds (or some fraction or multiple thereof). Otherwise,
the timezone adjustment doesn't make any sense.

So this isn't a case of limited functionality, just that we need to
inform the user that a timestamptz range with granularity '1 day' or '1
month' makes no sense.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] XLogInsert

[Andres Freund]

On Wednesday 16 December 2009 20:07:07 Gurjeet Singh wrote:
> 2009/12/15 Greg Smith 
> 
> > Jaime Casanova wrote:
> >> So in this extreme case avg tps is just 6 transactions better
> >
> > Great job trying to find the spot where the code worked better.  I'm not
> > so sure I trust pgbench results where the TPS was so low though.  Which
> > leads us right back to exactly how Jeff measured his original results.
> >
> > As I said already, I think we need more insight into Jeff's performance
> > report, a way to replicate that test, to look a bit at the latency as
> > reported by the updated LWLock patch that Pierre submitted.  Tweaking
> > your test to give more useful results is a nice second opinion on top of
> > that. But we're out of time for now, so this patch is getting returned
> > with feedback.  I encourage Jeff to resubmit the same patch or a better
> > one with a little more data on performance measurements to our final 8.5
> > CommitFest in hopes we can confirm this an improvement worth committing.
> 
> Last week I worked on a FUSE based filesystem, which I call BlackholeFS.
>  Its similar to /dev/null, but for directories. Basically it simply returns
>  success for all the writes, but doesn't do any writes on the files under
>  it.
I doubt that it will be faster than a tmpfs - the additional context switches 
et al probably will hurt already.
If you constrain the checkpoint_segments to something sensible it shouldnt use 
too much memory.


Andres

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] XLogInsert

[Gurjeet Singh]

2009/12/15 Greg Smith 

> Jaime Casanova wrote:
>
>> So in this extreme case avg tps is just 6 transactions better
>>
>>
> Great job trying to find the spot where the code worked better.  I'm not so
> sure I trust pgbench results where the TPS was so low though.  Which leads
> us right back to exactly how Jeff measured his original results.
>
> As I said already, I think we need more insight into Jeff's performance
> report, a way to replicate that test, to look a bit at the latency as
> reported by the updated LWLock patch that Pierre submitted.  Tweaking your
> test to give more useful results is a nice second opinion on top of that.
>  But we're out of time for now, so this patch is getting returned with
> feedback.  I encourage Jeff to resubmit the same patch or a better one with
> a little more data on performance measurements to our final 8.5 CommitFest
> in hopes we can confirm this an improvement worth committing.
>
>
>
Last week I worked on a FUSE based filesystem, which I call BlackholeFS. Its
similar to /dev/null, but for directories. Basically it simply returns
success for all the writes, but doesn't do any writes on the files under it.

Would moving the pg_xlog/ (and possibly table data too) to such a filesystem
exercise this patch better?

Best regards,
-- 
Lets call it Postgres

EnterpriseDB  http://www.enterprisedb.com

gurjeet[.sin...@enterprisedb.com

singh.gurj...@{ gmail | hotmail | indiatimes | yahoo }.com
Twitter: singh_gurjeet
Skype: singh_gurjeet

Mail sent from my BlackLaptop device

Re: [HACKERS] Range types

[Tom Lane]

Scott Bailey  writes:
> As I pointed out off-list, I think the granularity for timestamp range 
> should be limited to hours and smaller. Anything larger is asking for 
> trouble. And quite honestly if they wanted day granularity, they should 
> use date range.

I'm still not real clear on what the expected use-case is for this.
You're evidently envisioning applications where the allowed form of
an interval is constrained, but in the cases I can think of, the
constraints are a lot more convoluted than what you're proposing.
For example, if you're trying to do classroom scheduling, it might be
useful to constrain the periods to start and end on hour boundaries
--- but the next thing you'll want is to have it know that the "next"
slot after 5pm Friday is 8am Monday.  Except on holidays.  And then
there's the fact that my alma mater starts most hour-long classes on
the half hour.

I think that wiring such constraints into the low-level datatype is
doomed to failure.  What you'd be better off with is a function that
finds the "next" period given a current period and some suitable
representation of the off-limits intervals.  The argument for having
granularity wired into the datatype seems to boil down to just space
savings.  I don't find that compelling enough to justify code
contortions and user-visible restrictions on functionality.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Scott Bailey]

Ok, silly question here. But how do you determine the length of a 
continuous range? By definition length of [a, b) and (a, b] = b-a. But 
what about (a,b) and [a,b]? Are we saying that because they are 
continuous, the difference between values included in the range and 
those excluded are so infinitesimally small so as not to matter? Thus 
length (a,b) == length [a,b] == length [a,b)? And if that is the case, 
does the inclusiveness of the range really even matter?


And can anyone point me to a reference for working with continuous 
ranges? Google just insists that I spelled contiguous wrong.


Scott

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Kevin Grittner]

Alvaro Herrera  wrote:
 
> So you'd have to disable HOT updates when true serializability was
> active?
 
I wouldn't think so; but someone familiar with HOT logic could
probably determine whether the unmodified algorithm could be used by
reviewing the "simplifying assumptions" near the bottom of page 42,
and the page about "Generalizing to other database engines" (section
4.8).  I think those portions might stand on their own without
reading the rest of the document.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Boszormenyi Zoltan]

Robert Haas írta:
> On Wed, Dec 16, 2009 at 1:25 PM, Robert Haas  wrote:
>   
>> On Wed, Dec 16, 2009 at 1:14 PM, Alvaro Herrera
>>  wrote:
>> 
>>> Robert Haas escribió:
>>>   
 On Wed, Dec 16, 2009 at 10:29 AM, Andres Freund  wrote:
 
> On Wednesday 16 December 2009 16:24:42 Robert Haas wrote:
>   
>>>   Inserts and deletes follow the same protocol, obtaining an exclusive
>>>   lock on the row after the one being inserted or deleted. The result
>>>   of this locking protocol is that a range scan prevents concurrent
>>>   inserts or delete within the range of the scan, and vice versa.
>>>
>>> That sounds like it should actually work.
>>>   
>> Only if you can guarantee that the database will access the rows using
>> some particular index.  If it gets to the data some other way it might
>> accidentally circumvent the lock.  That's kind of a killer in terms of
>> making this work for PostgreSQL.
>> 
> Isnt the whole topic only relevant for writing access? There you have to
> access the index anyway.
>   
 Yeah, I guess you have to insert the new tuple.  I guess while you
 were at it you might check whether the next tuple is locked...
 
>>> So you'd have to disable HOT updates when true serializability was
>>> active?
>>>   
>> I thought about that, but I don't think so.   HOT only applies to
>> updates, and predicate locking only applies to inserts.  Unless I have
>> my head in the sand?
>> 
>
> Err, no, wait.  Predicate locking can apply to updates, but since HOT
> updates never update an indexed column, I think we might still be OK?
>   

A predicate can include columns from an index plus others.
Am I missing something?

> ...Robert
>
>   


-- 
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

--
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Robert Haas]

On Wed, Dec 16, 2009 at 1:25 PM, Robert Haas  wrote:
> On Wed, Dec 16, 2009 at 1:14 PM, Alvaro Herrera
>  wrote:
>> Robert Haas escribió:
>>> On Wed, Dec 16, 2009 at 10:29 AM, Andres Freund  wrote:
>>> > On Wednesday 16 December 2009 16:24:42 Robert Haas wrote:
>>> >> >   Inserts and deletes follow the same protocol, obtaining an exclusive
>>> >> >   lock on the row after the one being inserted or deleted. The result
>>> >> >   of this locking protocol is that a range scan prevents concurrent
>>> >> >   inserts or delete within the range of the scan, and vice versa.
>>> >> >
>>> >> > That sounds like it should actually work.
>>> >>
>>> >> Only if you can guarantee that the database will access the rows using
>>> >> some particular index.  If it gets to the data some other way it might
>>> >> accidentally circumvent the lock.  That's kind of a killer in terms of
>>> >> making this work for PostgreSQL.
>>> > Isnt the whole topic only relevant for writing access? There you have to
>>> > access the index anyway.
>>>
>>> Yeah, I guess you have to insert the new tuple.  I guess while you
>>> were at it you might check whether the next tuple is locked...
>>
>> So you'd have to disable HOT updates when true serializability was
>> active?
>
> I thought about that, but I don't think so.   HOT only applies to
> updates, and predicate locking only applies to inserts.  Unless I have
> my head in the sand?

Err, no, wait.  Predicate locking can apply to updates, but since HOT
updates never update an indexed column, I think we might still be OK?

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Robert Haas]

On Wed, Dec 16, 2009 at 1:14 PM, Alvaro Herrera
 wrote:
> Robert Haas escribió:
>> On Wed, Dec 16, 2009 at 10:29 AM, Andres Freund  wrote:
>> > On Wednesday 16 December 2009 16:24:42 Robert Haas wrote:
>> >> >   Inserts and deletes follow the same protocol, obtaining an exclusive
>> >> >   lock on the row after the one being inserted or deleted. The result
>> >> >   of this locking protocol is that a range scan prevents concurrent
>> >> >   inserts or delete within the range of the scan, and vice versa.
>> >> >
>> >> > That sounds like it should actually work.
>> >>
>> >> Only if you can guarantee that the database will access the rows using
>> >> some particular index.  If it gets to the data some other way it might
>> >> accidentally circumvent the lock.  That's kind of a killer in terms of
>> >> making this work for PostgreSQL.
>> > Isnt the whole topic only relevant for writing access? There you have to
>> > access the index anyway.
>>
>> Yeah, I guess you have to insert the new tuple.  I guess while you
>> were at it you might check whether the next tuple is locked...
>
> So you'd have to disable HOT updates when true serializability was
> active?

I thought about that, but I don't think so.   HOT only applies to
updates, and predicate locking only applies to inserts.  Unless I have
my head in the sand?

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Scott Bailey]

Tom Lane wrote:

Jeff Davis  writes:

[ hacky special-case representation for discrete timestamp ranges ]


I'm still not exactly clear on what the use-case is for discrete
timestamp ranges, and I wonder how many people are going to be happy
with a representation that can't handle a range that's open-ended
on the left.


They wouldn't. But the timestamp data would be the anchor, not 
necessarily the start point. As long as we ranges unbounded on both ends 
we'd be ok.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Alvaro Herrera]

Robert Haas escribió:
> On Wed, Dec 16, 2009 at 10:29 AM, Andres Freund  wrote:
> > On Wednesday 16 December 2009 16:24:42 Robert Haas wrote:
> >> >   Inserts and deletes follow the same protocol, obtaining an exclusive
> >> >   lock on the row after the one being inserted or deleted. The result
> >> >   of this locking protocol is that a range scan prevents concurrent
> >> >   inserts or delete within the range of the scan, and vice versa.
> >> >
> >> > That sounds like it should actually work.
> >>
> >> Only if you can guarantee that the database will access the rows using
> >> some particular index.  If it gets to the data some other way it might
> >> accidentally circumvent the lock.  That's kind of a killer in terms of
> >> making this work for PostgreSQL.
> > Isnt the whole topic only relevant for writing access? There you have to
> > access the index anyway.
> 
> Yeah, I guess you have to insert the new tuple.  I guess while you
> were at it you might check whether the next tuple is locked...

So you'd have to disable HOT updates when true serializability was
active?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Scott Bailey]

Jeff Davis wrote:

On Sun, 2009-12-13 at 23:49 -0800, Scott Bailey wrote:
So basically I have an anyrange pseudo type with the functions prev, 
next, last, etc defined. So instead of hard coding range types, we would 
allow the user to define their own range types. Basically if we are able 
to determine the previous and next values of the base types we'd be able 
to define a range type. I'm envisioning in a manner much like defining 
an enum type.


After an off-list discussion with Scott, I think there may be a solution
here that works for everyone if we don't try so hard to unify the
implementation of discrete and continuous ranges. The API should be very
similar, of course, but the implementation doesn't need to be.

Continuous ranges absolutely require the following information: start,
end, and inclusivity information.

But discrete ranges can instead be stored by counting the number of
granules from the start point. For instance, it could be stored as:
start, num_granules.

That has a lot of benefits for discrete ranges of time. First of all, it
allows the algebra to work reasonably well for the "days" and "months"
part of the interval, so we can allow a granule of 1 day/week/month/year
for a timestamp range. For output of the range, we can then just
multiply the granule by the number of granules, and add that to the
start time; thus avoiding the "incremental addition" problem with date
math. I think this works reasonably well for timestamp/date ranges --
let me know if there is a problem here (aside from timestamptz, which I
address below).

Secondly, in the case of a timestamp range, we can use 7 bytes for
storing the number of granules rather than another full 8-byte
timestamp, leaving one byte for flags to represent NULL boundaries,
infinite boundaries, etc. For timestamps that would still mean that an
interval could be 2000 years long with '1 microsecond' granularity. For
dates, 3 bytes is sufficient for a date range 45000 years long with
granules of '1 day'. That means that we can get back down to a 16 byte
representation for timestamp ranges, or 8 byte representation for date
ranges. There are a few details, like infinite ranges, but those can be
pretty easily solved with flags as well.

There's one problem, and that's for timestamptz ranges with intervals
that include days and months. Timezone adjustments are just not
well-defined for that kind of granule (nor would it be particularly
useful even if it magically worked), so this would have to be blocked
somehow. I think that's a special case, and we could provide the user
with a nice error message telling the user to use a date or timestamp
range instead.

So, the idea is to default to a continuous range type, but if the user
supplies a granule, prior and next functions, and other necessary
details, then it becomes a discrete range type.

 * continuous ranges can still have everything that everyone wants, 
   including flags to indicate special values.
 * discrete range granule is specified explicitly, so it's not an 
   "implementation detail"

 * discrete ranges can have a compact representation
 * discrete ranges would still have room for flags to indicate special 
   values
 
Comments?


As I pointed out off-list, I think the granularity for timestamp range 
should be limited to hours and smaller. Anything larger is asking for 
trouble. And quite honestly if they wanted day granularity, they should 
use date range. Also, I think the granule should be same type as 
returned when subtracting two subtypes. So granule of date range should 
be int not interval. And if user wanted something with month 
granularity, perhaps an enum range of 'MM' would be better.


Quite honestly the following 3 cases would probably meet 99% of need:
CREATE TYPE period AS RANGE(timestamptz(0), interval '1 s');
CREATE TYPE period AS RANGE(timestamptz(3), interval '1 ms');
CREATE TYPE period AS RANGE(timestamptz, interval '1 us');

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Tom Lane]

Jeff Davis  writes:
> [ hacky special-case representation for discrete timestamp ranges ]

I'm still not exactly clear on what the use-case is for discrete
timestamp ranges, and I wonder how many people are going to be happy
with a representation that can't handle a range that's open-ended
on the left.

> So, the idea is to default to a continuous range type, but if the user
> supplies a granule, prior and next functions, and other necessary
> details, then it becomes a discrete range type.

Huh?  You're not going to be able to have a special case data
representation for one or two data types at the same time as you have a
function-based datatype-independent concept of a parameterized range
type.  Well, maybe you could have special code paths for just date and
timestamp but it'd be horrid.

More importantly, the notion of a representation granule is still 100%
wishful thinking for any inexact-representation datatype, which is going
to be a severe crimp in getting this accepted for timestamp, let alone
defining it in a way that would allow users to try to apply it to
floats.  Float timestamps might not be the default case anymore but they
are still supported.

I think you should let go of the feeling that you have to shave bytes
off the storage format.  You're creating a whole lot of work for
yourself and a whole lot of user-visible corner cases in return for
what ultimately isn't much.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Robert Haas]

On Wed, Dec 16, 2009 at 12:31 PM, Jeff Davis  wrote:
> There's one problem, and that's for timestamptz ranges with intervals
> that include days and months. Timezone adjustments are just not
> well-defined for that kind of granule (nor would it be particularly
> useful even if it magically worked), so this would have to be blocked
> somehow. I think that's a special case, and we could provide the user
> with a nice error message telling the user to use a date or timestamp
> range instead.

This seems like a fairly special-purpose type.  You'd be targeting it
at people who are very concerned with storing large numbers of these
(so they really care about space consumption) but for some reason
don't need to mix days and months (actually, the current interval
representation stores days, months, and seconds separately).  I
certainly think this might be useful to some people but it doesn't
really sounds like a general range type facility, since it seems to
involve some hacks that are fairly datatype-specific.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Jeff Davis]

On Sun, 2009-12-13 at 23:49 -0800, Scott Bailey wrote:
> So basically I have an anyrange pseudo type with the functions prev, 
> next, last, etc defined. So instead of hard coding range types, we would 
> allow the user to define their own range types. Basically if we are able 
> to determine the previous and next values of the base types we'd be able 
> to define a range type. I'm envisioning in a manner much like defining 
> an enum type.

After an off-list discussion with Scott, I think there may be a solution
here that works for everyone if we don't try so hard to unify the
implementation of discrete and continuous ranges. The API should be very
similar, of course, but the implementation doesn't need to be.

Continuous ranges absolutely require the following information: start,
end, and inclusivity information.

But discrete ranges can instead be stored by counting the number of
granules from the start point. For instance, it could be stored as:
start, num_granules.

That has a lot of benefits for discrete ranges of time. First of all, it
allows the algebra to work reasonably well for the "days" and "months"
part of the interval, so we can allow a granule of 1 day/week/month/year
for a timestamp range. For output of the range, we can then just
multiply the granule by the number of granules, and add that to the
start time; thus avoiding the "incremental addition" problem with date
math. I think this works reasonably well for timestamp/date ranges --
let me know if there is a problem here (aside from timestamptz, which I
address below).

Secondly, in the case of a timestamp range, we can use 7 bytes for
storing the number of granules rather than another full 8-byte
timestamp, leaving one byte for flags to represent NULL boundaries,
infinite boundaries, etc. For timestamps that would still mean that an
interval could be 2000 years long with '1 microsecond' granularity. For
dates, 3 bytes is sufficient for a date range 45000 years long with
granules of '1 day'. That means that we can get back down to a 16 byte
representation for timestamp ranges, or 8 byte representation for date
ranges. There are a few details, like infinite ranges, but those can be
pretty easily solved with flags as well.

There's one problem, and that's for timestamptz ranges with intervals
that include days and months. Timezone adjustments are just not
well-defined for that kind of granule (nor would it be particularly
useful even if it magically worked), so this would have to be blocked
somehow. I think that's a special case, and we could provide the user
with a nice error message telling the user to use a date or timestamp
range instead.

So, the idea is to default to a continuous range type, but if the user
supplies a granule, prior and next functions, and other necessary
details, then it becomes a discrete range type.

 * continuous ranges can still have everything that everyone wants, 
   including flags to indicate special values.
 * discrete range granule is specified explicitly, so it's not an 
   "implementation detail"
 * discrete ranges can have a compact representation
 * discrete ranges would still have room for flags to indicate special 
   values
 
Comments?

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL project policy compendium

[Greg Smith]

Kurt Harriman wrote:

On 12/16/2009 7:10 AM, Tom Lane wrote:

the project policy is to not require any compiler features not found

> in C89.

Is there somewhere a compendium of such policies which
fledgling hackers should consult to avoid embarrassment?


The list of suggestions at http://wiki.postgresql.org/wiki/Developer_FAQ 
and http://wiki.postgresql.org/wiki/Submitting_a_Patch are what we've 
got now.  There's a number of us who try to record any interesting 
discussion bits like this one into one of those, to shorten the future 
learning curve for others.


--
Greg Smith2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
g...@2ndquadrant.com  www.2ndQuadrant.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Tom Lane]

Alvaro Herrera  writes:
> Tom Lane escribió:
>> I think you're way overthinking this.  Where we started was just
>> a proposal to try to expand the set of inline-ing compilers beyond
>> "gcc only".  I don't see why we need to do anything but that.  The
>> code is fine as-is except for the control #ifdefs.

> IIRC Kurt was also on about getting rid of some ugly macros that could
> instead be coded as inline functions (fastgetattr for example)

I'd just bounce that as useless activity.  If they are macros now,
and work, the only possible effects of changing them are negative.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [ADMIN] recovery is stuck when children are not processing SIGQUIT from previous crash

[Tom Lane]

Peter Eisentraut  writes:
> Here is a set of patches to address this issue.
> The first one is a small refactoring of the signal setting portability
> business.

OK

> The second one fixes the SIGQUIT handler inadvertently unblocking
> SIGQUIT within itself.

OK

> The third one installs an alarm so that if the ereport() call in
> quickdie() doesn't finish after 60 seconds, it skips it and finishes up.
> The precise logic of this could be debated, but it more or less appears
> to get the job done.

I'm not too happy with depending on alarm(), which according to the
pgbench sources is not portable to Windows.  The postmaster does
something equivalent using enable_sig_alarm(); can we use that?

A different line of thought is that this still doesn't fix the problem
because you're depending on a fair amount of libc infrastructure that
might not be too reentrant (and the same objection could probably be
mounted against enable_sig_alarm).  It would be better for the
postmaster to be in charge of enforcing the timeout, and have it issue
SIGKILL against children that don't die fast enough to suit it.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Marko Kreen]

On 12/16/09, Tom Lane  wrote:
> Marko Kreen  writes:
>
> > So the plain-C89 compilers would be downgraded to "second-class"
>  > targets, not worth getting max performance out of them.
>
>
>  Hm?  Failing to inline is already a performance hit, which is why
>  Kurt got interested in this in the first place.
>
>  I think you're way overthinking this.  Where we started was just
>  a proposal to try to expand the set of inline-ing compilers beyond
>  "gcc only".  I don't see why we need to do anything but that.  The
>  code is fine as-is except for the control #ifdefs.

My proposal is basically about allowing more widespread use of
"static inline".  That is - "static inline" does not need to be
paired with equivalent macro.

But if C89 compilers are still project's primary target, then this
cannot be allowed.

-- 
marko

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Tom Lane]

Marko Kreen  writes:
> On 12/15/09, Kurt Harriman  wrote:
>> Attached is a revised patch, offered for the 2010-01 commitfest.
>> It's also available in my git repository in the "submitted" branch:
>> 
>> http://git.postgresql.org/gitweb?p=users/harriman/share.git;a=shortlog;h=refs/heads/submitted

> -1.  The PG_INLINE is ugly.

FWIW, I think the patch is largely OK, except for the autoconf hackery
which I'm not the best-qualified person to opine on.  I would only
suggest that the cleanest coding would be

#ifdef USE_INLINE

static inline foo(...) ...

#else

... non-inline definition of foo

#endif

ie, go ahead and rely on autoconf's definition (if any) of "inline"
and add a policy symbol USE_INLINE to determine whether to use it.
The proposed PG_INLINE coding conflates the symbol needed in the code
with the policy choice.

Another possibility would be to call the policy symbol HAVE_INLINE,
but that (a) risks collision with a name defined by autoconf built-in
macros, and (b) looks like it merely indicates whether the compiler
*has* inline, not that we have made a choice about how to use it.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] idea - new aggregates median, listagg

[Scott Bailey]

Thom Brown wrote:
2009/12/15 Pavel Stehule >


Hello

I am looking on new feature - ORDER clause in aggregate, and I thing,
so we are able to effectively implement some non standard, but well
known aggregates.

a) function median - it is relative frequent request - with usually
slow implementation

b) function listagg (it is analogy of group_concat from MySQL) - it
should simplify report generating and some other

What is your opinion? Do you like to see these functions in core?



I'm probably missing the point here, but when I originally saw MySQL's 
group_concat function, I found it odd that it featured ordering 
functionality.  Shouldn't the order by determined by the query itself?  
Otherwise it's almost as if its separating the relationship between the 
result column and the resultset.


For xmlagg in particular, it is quite useful to be able order the 
results. And sorting the query doesn't work for ordering the agg unless 
you do it in a subquery. Oracle has this functionality and it is quite 
handy. It would be nice to see listagg with the option to order as well.


Scott

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Alvaro Herrera]

Tom Lane escribió:

> I think you're way overthinking this.  Where we started was just
> a proposal to try to expand the set of inline-ing compilers beyond
> "gcc only".  I don't see why we need to do anything but that.  The
> code is fine as-is except for the control #ifdefs.

IIRC Kurt was also on about getting rid of some ugly macros that could
instead be coded as inline functions (fastgetattr for example)

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL project policy compendium

[Tom Lane]

Robert Haas  writes:
> On Wed, Dec 16, 2009 at 10:44 AM, Kurt Harriman  wrote:
>> Is there somewhere a compendium of such policies which
>> fledgling hackers should consult to avoid embarrassment?

> I do think your basic point is well-taken, though.  There are a lot of
> details about how we do things that I have had to learn through
> osmosis, and maybe if I can figure out what they all are, I'll try to
> write them down somewhere, if someone else hasn't done it already.

Yeah, this is something that is difficult for the senior hackers to
undertake because we've forgotten what it was we had to learn the hard
way ;-).  Maybe some of the newer project members could collaborate
on scribbling down such stuff (wiki page?).

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [ADMIN] recovery is stuck when children are not processing SIGQUIT from previous crash

[Alvaro Herrera]

Peter Eisentraut escribió:
> Here is a set of patches to address this issue.
> 
> The first one is a small refactoring of the signal setting portability
> business.

This one looks like should be applied immediately to get some buildfarm
coverage (and alpha3)



-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL project policy compendium

[Robert Haas]

On Wed, Dec 16, 2009 at 10:44 AM, Kurt Harriman  wrote:
> On 12/16/2009 7:10 AM, Tom Lane wrote:
>>
>> the project policy is to not require any compiler features not found
>
>> in C89.
>
> Is there somewhere a compendium of such policies which
> fledgling hackers should consult to avoid embarrassment?

Well the best way to avoid embarrassment is to just not be
embarrassed.  I don't think anyone will think ill of you for
suggesting something that turns out not to be where the project wants
to go, as long as you're not too dogmatic about it.

This particular point is mentioned in the documentation:

http://www.postgresql.org/docs/8.4/static/install-requirements.html

The rest of chapter 15 is worth a read, too.

I do think your basic point is well-taken, though.  There are a lot of
details about how we do things that I have had to learn through
osmosis, and maybe if I can figure out what they all are, I'll try to
write them down somewhere, if someone else hasn't done it already.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL project policy compendium

[Peter Eisentraut]

On ons, 2009-12-16 at 07:44 -0800, Kurt Harriman wrote: 
> On 12/16/2009 7:10 AM, Tom Lane wrote:
> > the project policy is to not require any compiler features not found
>  > in C89.
> 
> Is there somewhere a compendium of such policies which
> fledgling hackers should consult to avoid embarrassment?

The installation instructions say that you need a C89 compiler.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Tom Lane]

Marko Kreen  writes:
> So the plain-C89 compilers would be downgraded to "second-class"
> targets, not worth getting max performance out of them.

Hm?  Failing to inline is already a performance hit, which is why
Kurt got interested in this in the first place.

I think you're way overthinking this.  Where we started was just
a proposal to try to expand the set of inline-ing compilers beyond
"gcc only".  I don't see why we need to do anything but that.  The
code is fine as-is except for the control #ifdefs.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [ADMIN] recovery is stuck when children are not processing SIGQUIT from previous crash

[Peter Eisentraut]

Here is a set of patches to address this issue.

The first one is a small refactoring of the signal setting portability
business.

The second one fixes the SIGQUIT handler inadvertently unblocking
SIGQUIT within itself.

The third one installs an alarm so that if the ereport() call in
quickdie() doesn't finish after 60 seconds, it skips it and finishes up.
The precise logic of this could be debated, but it more or less appears
to get the job done.
>From d70d637ffe60dd3c73690a2640e84ad6e2cdbb50 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 16 Dec 2009 14:41:10 +0200
Subject: [PATCH 1/3] If there is no sigdelset(), define it as a macro

Removes some duplicate code that recreated the identical workaround.
---
 src/backend/postmaster/bgwriter.c  |4 
 src/backend/postmaster/walwriter.c |4 
 src/backend/tcop/postgres.c|4 
 src/include/libpq/pqsignal.h   |6 --
 4 files changed, 4 insertions(+), 14 deletions(-)

diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index a2c32a7..81c0d36 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -244,11 +244,7 @@ BackgroundWriterMain(void)
 	pqsignal(SIGWINCH, SIG_DFL);
 
 	/* We allow SIGQUIT (quickdie) at all times */
-#ifdef HAVE_SIGPROCMASK
 	sigdelset(&BlockSig, SIGQUIT);
-#else
-	BlockSig &= ~(sigmask(SIGQUIT));
-#endif
 
 	/*
 	 * Initialize so that first time-driven event happens at the correct time.
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 90b8cfe..7d95696 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -126,11 +126,7 @@ WalWriterMain(void)
 	pqsignal(SIGWINCH, SIG_DFL);
 
 	/* We allow SIGQUIT (quickdie) at all times */
-#ifdef HAVE_SIGPROCMASK
 	sigdelset(&BlockSig, SIGQUIT);
-#else
-	BlockSig &= ~(sigmask(SIGQUIT));
-#endif
 
 	/*
 	 * Create a resource owner to keep track of our resources (not clear that
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c985478..c7906bf 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3292,11 +3292,7 @@ PostgresMain(int argc, char *argv[], const char *username)
 	if (IsUnderPostmaster)
 	{
 		/* We allow SIGQUIT (quickdie) at all times */
-#ifdef HAVE_SIGPROCMASK
 		sigdelset(&BlockSig, SIGQUIT);
-#else
-		BlockSig &= ~(sigmask(SIGQUIT));
-#endif
 	}
 
 	PG_SETMASK(&BlockSig);		/* block everything except SIGQUIT */
diff --git a/src/include/libpq/pqsignal.h b/src/include/libpq/pqsignal.h
index d8e791c..82414d6 100644
--- a/src/include/libpq/pqsignal.h
+++ b/src/include/libpq/pqsignal.h
@@ -26,7 +26,7 @@ extern sigset_t UnBlockSig,
 			StartupBlockSig;
 
 #define PG_SETMASK(mask)	sigprocmask(SIG_SETMASK, mask, NULL)
-#else
+#else /* not HAVE_SIGPROCMASK */
 extern int	UnBlockSig,
 			BlockSig,
 			StartupBlockSig;
@@ -37,7 +37,9 @@ extern int	UnBlockSig,
 #define PG_SETMASK(mask)		pqsigsetmask(*((int*)(mask)))
 int			pqsigsetmask(int mask);
 #endif
-#endif
+
+#define sigdelset(set, signum)	(*(set) &= ~(sigmask(signum)))
+#endif /* not HAVE_SIGPROCMASK */
 
 typedef void (*pqsigfunc) (int);
 
-- 
1.6.5

>From 96a3bc5fe46e226ff179a729baa58efc16c0aaba Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 16 Dec 2009 16:07:12 +0200
Subject: [PATCH 2/3] Don't unblock SIGQUIT in the SIGQUIT handler

---
 src/backend/tcop/postgres.c  |1 +
 src/include/libpq/pqsignal.h |1 +
 2 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c7906bf..b2fb501 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -2538,6 +2538,7 @@ drop_unnamed_stmt(void)
 void
 quickdie(SIGNAL_ARGS)
 {
+	sigaddset(&BlockSig, SIGQUIT); /* prevent nested calls */
 	PG_SETMASK(&BlockSig);
 
 	/*
diff --git a/src/include/libpq/pqsignal.h b/src/include/libpq/pqsignal.h
index 82414d6..4659ae2 100644
--- a/src/include/libpq/pqsignal.h
+++ b/src/include/libpq/pqsignal.h
@@ -38,6 +38,7 @@ extern int	UnBlockSig,
 int			pqsigsetmask(int mask);
 #endif
 
+#define sigaddset(set, signum)	(*(set) |= (sigmask(signum)))
 #define sigdelset(set, signum)	(*(set) &= ~(sigmask(signum)))
 #endif /* not HAVE_SIGPROCMASK */
 
-- 
1.6.5

>From bebb95abe7a55173cab0558da3373d6a3631465b Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 16 Dec 2009 17:19:14 +0200
Subject: [PATCH 3/3] Time out the ereport() call in quickdie() after 60 seconds

---
 src/backend/tcop/postgres.c |   25 +
 1 files changed, 25 insertions(+), 0 deletions(-)

diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index b2fb501..ab6805a 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -191,6 +191,7 @@ static bool IsTransactionExitStmtList(List *parseTrees);
 static bool IsTransactionStmtList(List *parseTrees);
 static void drop_unnamed_stmt(void);
 static void Sig

[HACKERS] PostgreSQL project policy compendium

[Kurt Harriman]

On 12/16/2009 7:10 AM, Tom Lane wrote:

the project policy is to not require any compiler features not found

> in C89.

Is there somewhere a compendium of such policies which
fledgling hackers should consult to avoid embarrassment?

Regards,
... kurt


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fast or immediate shutdown

[Simon Riggs]

On Wed, 2009-12-16 at 17:04 +0200, Peter Eisentraut wrote:
> On tis, 2009-12-15 at 17:19 +, Simon Riggs wrote:
> > running with log_checkpoints = on
> > 
> > pg_ctl -D foo -m fast stop
> > 
> > log says
> > 
> > LOG:  received fast shutdown request
> > LOG:  aborting any active transactions
> > LOG:  shutting down
> > LOG:  restartpoint starting: shutdown immediate
> > 
> > Some of us know that the "immediate" word refers to the restartpoint
> > request, though that word causes conceptual conflict with the shutdown
> > mode, which was fast, not immediate.
> > 
> > Think we need to change the wording of this
> > 
> > LOG:  restartpoint starting: shutdown immediate
> > 
> > so it is clearer what we mean
> 
> We *do* we mean?  And why are we logging it?

The words after the colon refer to options sent to RequestCheckpoint and
it is logged because we asked for it by doing log_checkpoints = on.

I suggest we say "smoothed" when checkpoint option is !immediate. So we
will remove the word "immediate" from the message.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] idea - new aggregates median, listagg

[Pavel Stehule]

2009/12/16 Thom Brown :
> 2009/12/15 Pavel Stehule 
>>
>> Hello
>>
>> I am looking on new feature - ORDER clause in aggregate, and I thing,
>> so we are able to effectively implement some non standard, but well
>> known aggregates.
>>
>> a) function median - it is relative frequent request - with usually
>> slow implementation
>>
>> b) function listagg (it is analogy of group_concat from MySQL) - it
>> should simplify report generating and some other
>>
>> What is your opinion? Do you like to see these functions in core?
>>
>>
>
> I'm probably missing the point here, but when I originally saw MySQL's
> group_concat function, I found it odd that it featured ordering
> functionality.  Shouldn't the order by determined by the query itself?
> Otherwise it's almost as if its separating the relationship between the
> result column and the resultset.
>

Aggregates as group_concat or listagg are not typical SQL aggregates.
With these aggregates we are able to do some reports on SQL level
without stored procedures. What I know, order is determined only for
non hash aggregates - and you cannot specify method of aggregation, so
possibility to specify ORDER is important. But this feature isn't
related to this "proposal". It was commited yesterday - so you can
look on discussion about this feature.

Regards
Pavel Stehuke

> Thom
>
>

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Kevin Grittner]

Nicolas Barbier  wrote:
 
> I am not sure whether the serialization failures that it may cause
> are dependent on the plan used.
 
They are.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Kevin Grittner]

"Albe Laurenz"  wrote:
 
> Although it may have seemed that I was out to shoot the idea down,
> I am interested in the topic. I guess my way of understanding
> something is trying to find holes in it...
 
No problem.  That's how ideas are explored and improved.  The brick
wall was that there seemed to be overwhelming opposition to any new
technique which causes serialization failures under conditions where
the cause isn't blindingly obvious, regardless of the resulting
improvements in data reliability.
 
> I was particularly interested how he solved the problem of phantom
> reads.
 
> That sounds like it should actually work.
 
Well, that's certainly not the novel part -- those techniques were
described and proven theoretically decades ago and have been in use
by many popular products for almost as long.  It's that non-blocking
SIREAD lock which is the fun part.  ;-)
 
I just wish I had time to read all the documents referenced in the
footnotes
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Marko Kreen]

On 12/16/09, Tom Lane  wrote:
> Marko Kreen  writes:
>  > Hypothetical old, crappy compilers would still work, only AC_C_INLINE
>  > would turn "static inline" into plain "static", so hypothetically
>  > they would get some warnings about unused functions.
>
>  > As this is all hypothetical, I don't see why that should stop us
>  > cleaning our code?
>
>  There's nothing "hypothetical" about it --- I still regularly check
>  that the code builds on an old HP compiler that doesn't have inline.

Ok, good.  Thus far only argument was "historically they have existed",
which does not sound good enough worry about them.  If somebody is
actually testing and caring abouth such compilers, they need to be
taken more seriously.

>  I remind you that the project policy is to not require any compiler
>  features not found in C89.  If you can exploit inline on more compilers
>  than now, fine, but assuming that everything has got it is not OK.

Note - my advanced proposal (instead duplicate macros, let 'static inline'
functions fall back to being plain 'static') would still support
non-inline compilers, but with potentially small performance hit.

So the plain-C89 compilers would be downgraded to "second-class"
targets, not worth getting max performance out of them.

Is this OK?

-- 
marko

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Robert Haas]

On Wed, Dec 16, 2009 at 10:29 AM, Andres Freund  wrote:
> On Wednesday 16 December 2009 16:24:42 Robert Haas wrote:
>> >   Inserts and deletes follow the same protocol, obtaining an exclusive
>> >   lock on the row after the one being inserted or deleted. The result
>> >   of this locking protocol is that a range scan prevents concurrent
>> >   inserts or delete within the range of the scan, and vice versa.
>> >
>> > That sounds like it should actually work.
>>
>> Only if you can guarantee that the database will access the rows using
>> some particular index.  If it gets to the data some other way it might
>> accidentally circumvent the lock.  That's kind of a killer in terms of
>> making this work for PostgreSQL.
> Isnt the whole topic only relevant for writing access? There you have to
> access the index anyway.

Yeah, I guess you have to insert the new tuple.  I guess while you
were at it you might check whether the next tuple is locked...

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Update on true serializable techniques in MVCC

[Andres Freund]

Moin,

On Wednesday 16 December 2009 16:24:42 Robert Haas wrote:
> >   Inserts and deletes follow the same protocol, obtaining an exclusive
> >   lock on the row after the one being inserted or deleted. The result
> >   of this locking protocol is that a range scan prevents concurrent
> >   inserts or delete within the range of the scan, and vice versa.
> >
> > That sounds like it should actually work.
> 
> Only if you can guarantee that the database will access the rows using
> some particular index.  If it gets to the data some other way it might
> accidentally circumvent the lock.  That's kind of a killer in terms of
> making this work for PostgreSQL.
Isnt the whole topic only relevant for writing access? There you have to 
access the index anyway.

Andres

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] idea - new aggregates median, listagg

[Thom Brown]

2009/12/15 Pavel Stehule 

> Hello
>
> I am looking on new feature - ORDER clause in aggregate, and I thing,
> so we are able to effectively implement some non standard, but well
> known aggregates.
>
> a) function median - it is relative frequent request - with usually
> slow implementation
>
> b) function listagg (it is analogy of group_concat from MySQL) - it
> should simplify report generating and some other
>
> What is your opinion? Do you like to see these functions in core?
>
>
>
I'm probably missing the point here, but when I originally saw MySQL's
group_concat function, I found it odd that it featured ordering
functionality.  Shouldn't the order by determined by the query itself?
Otherwise it's almost as if its separating the relationship between the
result column and the resultset.

Thom

Re: [HACKERS] Update on true serializable techniques in MVCC

[Robert Haas]

On Wed, Dec 16, 2009 at 4:52 AM, Albe Laurenz  wrote:
> Kevin Grittner wrote:
>> Just to make those who care aware of it, here is Michael Cahill's
>> Doctoral Thesis based on implementing Serializable Snapshot
>> Isolation in InnoDB using a refined version of the techniques
>> previously used in the Berkley DB (and previously discussed on this
>> list):
>>
>> http://hdl.handle.net/2123/5353
>>
>> Seriously, this post is just for the benefit of those who may be
>> interested in following these developments -- I don't have the
>> inclination or energy for another round of debate on the topic just
>> now.  :-/
>
> I understand that, and thank you for the information.
> Although it may have seemed that I was out to shoot the idea down,
> I am interested in the topic. I guess my way of understanding something
> is trying to find holes in it...
>
> I read into the text, and I was particularly interested how he solved
> the problem of phantom reads.
>
> Quote:
>   The problem [of phantom reads] was identified in (Eswaran et al., 1976),
>   but the general purpose "predicate locking" solution suggested there
>   has not been widely adopted because of the difficulty in testing mutual
>   satisfiability of predicates.
>
>   Instead, locking DBMS implementations commonly use algorithms based on
>   "next-key locking". In these, a range of key space is protected against
>   concurrent insertion or deletion by acquiring a shared lock on the next
>   row in order, as a scan is made to check whether rows match a predicate.
>   The scan might be through the data records or through an index.
>
>   Inserts and deletes follow the same protocol, obtaining an exclusive
>   lock on the row after the one being inserted or deleted. The result
>   of this locking protocol is that a range scan prevents concurrent
>   inserts or delete within the range of the scan, and vice versa.
>
> That sounds like it should actually work.

Only if you can guarantee that the database will access the rows using
some particular index.  If it gets to the data some other way it might
accidentally circumvent the lock.  That's kind of a killer in terms of
making this work for PostgreSQL.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Marko Kreen]

On 12/16/09, Tom Lane  wrote:
> Marko Kreen  writes:
>  > On 12/16/09, Kurt Harriman  wrote:
>
> >> For gcc, I think the __attribute__ has to come after the function's
>  >> parameter list, rather than before the return type.
>
>  > No.
>
>
> [ squint... ]  That's nowhere documented that I can find: all the
>  examples in the gcc docs show __attribute__ after the parameters.
>  It does seem to work, but should we rely on it?

Heh.  At least in 3.0, 3.1 and 4.2 docs there is:

  __attribute__((noreturn)) void d0 (void),
  __attribute__((format(printf, 1, 2))) d1 (const char *, ...),
  d2 (void);

describing that _att before decl list applies to all declarations,
but _att after decl applies to only that declaration.  That sort of
explains also why all examples have _att afterwards.

  http://gcc.gnu.org/onlinedocs/gcc-3.1.1/gcc/Attribute-Syntax.html

But I dare not to pick out sentences from Function-Attributes.html
that describe that...

>  The bigger problem though is that not all versions of gcc understand
>  always_inline:
>
>  $ gcc -Wall check.c
>  check.c:3: warning: `always_inline' attribute directive ignored

Oh, another argument against logic in headers.

>  which I think is sufficient reason to put an end to this sub-thread.
>  We have no particular need for force-inline semantics anyway, as
>  long as the compiler behaves reasonably for unreferenced inlines,
>  which gcc always has.

Ok.

-- 
marko

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] idea - new aggregates median, listagg

[Kevin Grittner]

Pavel Stehule  wrote:
> 2009/12/15 David Fetter :
 
>> http://www.simple-talk.com/sql/t-sql-programming/median-workbench/
> 
> In this article the are two medians - statistical and financial. I
> am for both. But only one can be named "median".
 
Well, since the statistical median requires that you specify whether
you want the left or right median (in case there are an even number
of values), you could either have two median functions, one of which
would take an extra parameter for this, or you could have median,
left_median, and right_median functions.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Tom Lane]

Marko Kreen  writes:
> Hypothetical old, crappy compilers would still work, only AC_C_INLINE
> would turn "static inline" into plain "static", so hypothetically
> they would get some warnings about unused functions.

> As this is all hypothetical, I don't see why that should stop us
> cleaning our code?

There's nothing "hypothetical" about it --- I still regularly check
that the code builds on an old HP compiler that doesn't have inline.

I remind you that the project policy is to not require any compiler
features not found in C89.  If you can exploit inline on more compilers
than now, fine, but assuming that everything has got it is not OK.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fast or immediate shutdown

[Peter Eisentraut]

On tis, 2009-12-15 at 17:19 +, Simon Riggs wrote:
> running with log_checkpoints = on
> 
> pg_ctl -D foo -m fast stop
> 
> log says
> 
> LOG:  received fast shutdown request
> LOG:  aborting any active transactions
> LOG:  shutting down
> LOG:  restartpoint starting: shutdown immediate
> 
> Some of us know that the "immediate" word refers to the restartpoint
> request, though that word causes conceptual conflict with the shutdown
> mode, which was fast, not immediate.
> 
> Think we need to change the wording of this
> 
> LOG:  restartpoint starting: shutdown immediate
> 
> so it is clearer what we mean

We *do* we mean?  And why are we logging it?


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Robert Haas]

On Wed, Dec 16, 2009 at 9:30 AM, Marko Kreen  wrote:
> On 12/16/09, Robert Haas  wrote:
>> On Tue, Dec 15, 2009 at 10:34 PM, Kurt Harriman  wrote:
>>  >> Your worry ii) can be ignored, managing to compile on such
>>  >> compilers is already overachievement.
>>  >
>>  > I think so too.  With your opinion added to mine, do we constitute a
>>  > consensus of the pg community?  Someone might object that a sample of
>>  > two individuals is insufficiently representative of the whole, but
>>  > away with the pedants: let us not quibble over trifles.
>>
>>
>> I haven't completely followed this thread, but I think there has been
>>  some discussion of making changes to inline that would cause
>>  regressions for people using old, crappy compilers, and I think we
>>  should avoid doing that unless there is some compelling benefit.  I'm
>>  not sure what that benefit would be - I don't think "cleaner code" is
>>  enough.
>
> Seems you have not followed the thread...
>
> Hypothetical old, crappy compilers would still work, only AC_C_INLINE
> would turn "static inline" into plain "static", so hypothetically
> they would get some warnings about unused functions.
>
> As this is all hypothetical, I don't see why that should stop us
> cleaning our code?

I don't think that hypothetical is the right word.  There is no
question that such compilers exist.  What seems to me to be relevant
is whether anyone is still using them.  Maybe with sufficient research
you could demonstrate that there are no platforms we support where
these are still in common use - e.g. the oldest version of AIX
compiler that has this problem is version X, but PostgreSQL is only
supported on versions Y and higher, where Y>X.  But I don't see that
any attempt has been made to do that research, or at least I haven't
seen any attempt to go through our supported platforms and discuss the
situation for each one.

I'm not really sure there's enough benefit in this project to warrant
that effort, but that's up to you to decide.  However, I am pretty
confident that there is going to be opposition to degrading
performance or causing lots of compiler warnings on older platforms
that are still supported.  In fact, such opposition has already been
registered on this thread and you can find it here:

http://archives.postgresql.org/pgsql-hackers/2009-12/msg00239.php

Our standard for accepting patches is that (1) they have a
demonstrated benefit and (2) they don't break anything.  Leaving aside
the question of whether this patch makes anything worse, it would
really improve the case for this patch if someone could enumerate the
supported platforms and compilers where it makes things better.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[Tom Lane]

Dimitri Fontaine  writes:
> Tom Lane  writes:
>>  foreach p2_member in unnest(p2) loop
>>  p1 := array(select period_except(p1_member, p2_member)
>>  from unnest(p1) p1_member);
>>  end loop;
>> 
>> But maybe it can be done in a single SQL command.

> Yeah, as soon as you have LATERAL, I think. Without it there's no way to
> compose SRF in SQL, AFAIK.

Hm, how would you do it with LATERAL?  The problem is not so much
composition as the need for a variable number of rounds of composition.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Tom Lane]

Marko Kreen  writes:
> On 12/16/09, Kurt Harriman  wrote:
>> For gcc, I think the __attribute__ has to come after the function's
>> parameter list, rather than before the return type.

> No.

[ squint... ]  That's nowhere documented that I can find: all the
examples in the gcc docs show __attribute__ after the parameters.
It does seem to work, but should we rely on it?

The bigger problem though is that not all versions of gcc understand
always_inline:

$ gcc -Wall check.c
check.c:3: warning: `always_inline' attribute directive ignored

which I think is sufficient reason to put an end to this sub-thread.
We have no particular need for force-inline semantics anyway, as
long as the compiler behaves reasonably for unreferenced inlines,
which gcc always has.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] idea - new aggregates median, listagg

[Robert Haas]

On Wed, Dec 16, 2009 at 1:36 AM, Pavel Stehule  wrote:
>>> b) function listagg (it is analogy of group_concat from MySQL) - it
>>> should simplify report generating and some other
>>
>> This is redundant, as it's equivalent to array_to_string(array_agg()).
>
> when I implement it in orafce - the almost all code was parameters
> checking. Implementation is trivial, because important part of work
> was done with array_agg support. It is wrapper over stringInfo.
>
>>
>> If it's done as syntactic sugar over that, it's fine.  If it's a
>> separate implementation, it's a bad idea.  Probably best as an
>> optional module of some kind
>
> I am not sure if implementation as syntactic sugar is best. It needs
> parser changes.

I don't think this is important enough to change the parser for it.  I
don't see a concrete proposal for syntax here, but it seems like  the
standard function call syntax should be adequate.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Marko Kreen]

On 12/16/09, Robert Haas  wrote:
> On Tue, Dec 15, 2009 at 10:34 PM, Kurt Harriman  wrote:
>  >> Your worry ii) can be ignored, managing to compile on such
>  >> compilers is already overachievement.
>  >
>  > I think so too.  With your opinion added to mine, do we constitute a
>  > consensus of the pg community?  Someone might object that a sample of
>  > two individuals is insufficiently representative of the whole, but
>  > away with the pedants: let us not quibble over trifles.
>
>
> I haven't completely followed this thread, but I think there has been
>  some discussion of making changes to inline that would cause
>  regressions for people using old, crappy compilers, and I think we
>  should avoid doing that unless there is some compelling benefit.  I'm
>  not sure what that benefit would be - I don't think "cleaner code" is
>  enough.

Seems you have not followed the thread...

Hypothetical old, crappy compilers would still work, only AC_C_INLINE
would turn "static inline" into plain "static", so hypothetically
they would get some warnings about unused functions.

As this is all hypothetical, I don't see why that should stop us
cleaning our code?

-- 
marko

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

[Marko Kreen]

On 12/16/09, Kurt Harriman  wrote:
> [Please ignore the previous incomplete version of this reply, which I
>  sent by mistake.  Sorry for the list noise.]
>
>  On 12/15/2009 2:09 PM, Marko Kreen wrote:
>  >
>  > Oh.  Ok then.  Force-inline seems better fix as we may want to use
>  > it for other reasons too (converting big macros).
>  >
>  > So it seems like a good moment to solve it for gcc too.
>
>  Is ordinary inlining not sufficient?
>
>  If deluxe inlining is something that might be wanted in the
>  future, but isn't needed right now, perhaps we should wait
>  until then.

It allows the effects the __forceinline has on MSVC also observe
on GCC.  It should not hurt anything, unless we go overboard with
inline usage.

But yeah, it can be omitted.  I was just wondering on code structure,
to make it possible to turn forceinline for gcc too.

>  > Your worry ii) can be ignored, managing to compile on such
>  > compilers is already overachievement.
>
>  I think so too.  With your opinion added to mine, do we constitute a
>  consensus of the pg community?  Someone might object that a sample of
>  two individuals is insufficiently representative of the whole, but
>  away with the pedants: let us not quibble over trifles.

Note that we are not voting here, we are on technical discussion list.
Interested people can voice counter-arguments, if we can overturn them,
we can proceed...

>  > The question is now what should we put into configure and what into
>  > headers.
>
>  PostgreSQL seems mostly to follow the GNU tradition of using
>  autoconf rather than thickets of #ifdefs to discover platform
>  characteristics such as supported features, variant spellings of
>  keywords and identifiers, and the like.  This often makes it easier
>  to support new platforms, assuming the autoconf mechanism is
>  understood.
>
>  Autoconf facilitates testing directly for the needed features:
>  generally a better approach than hard-coding knowledge of the
>  peculiarities of particular compilers, compiler release levels,
>  instruction set architectures, etc.  For example, instead of
>  writing #ifdefs to decide, "if this is gcc, and the version is
>  high enough, then __funcname__ should work", it's better to let
>  autoconf actually try to compile a program using __funcname__.
>  That way PostgreSQL can keep up with the evolution of improved
>  and new compilers without continually updating an #ifdef-encoded
>  knowledge base of compiler capabilities.

Well, yeah, except on one platform we are mostly bypassing the
autoconf logic and have lot of duplicate logic.

But I'm not opposed putting it into configure, assuming the
final keyword is still called "inline".

>  > Simplest would be to have plain AC_C_INLINE in configure
>  > and then in header (c.h?):
>  >
>  > #ifdef _MSC_VER
>  > #undef inline
>  > #define inline __forceinline
>  > #else
>  > #ifdef __GNUC__
>  > #undef inline
>  > #define inline inline __attribute__((always_inline))
>  > #endif
>  > #endif
>  >
>  > (Not compile tested :)
>
>  This would force every inline.  Why do we want that?

The compiler is allowed to uninline a plain "inline" function
if it feels like it.  The gcc uses guesstimate instruction
count (before inlining) to decide on that issue.

ATM it's not a problem, but eg. if we want to turn more complex
macros into inlines (heap_getattr()), it may go over limit and get
uninlined.

>  For gcc, I think the __attribute__ has to come after the function's
>  parameter list, rather than before the return type.

No.


-- 
marko

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] An example of bugs for Hot Standby

[Simon Riggs]

On Wed, 2009-12-16 at 10:33 +, Simon Riggs wrote:
> On Tue, 2009-12-15 at 20:25 +0900, Hiroyuki Yamada wrote:
> > Hot Standby node can freeze when startup process calls 
> > LockBufferForCleanup().
> > This bug can be reproduced by the following procedure.
> 
> Interesting. Looks like this can happen, which is a shame cos I just
> removed the wait checking code after not ever having seen a wait.
> 
> Thanks for the report.
> 
> Must-fix item for HS.

So this deadlock can happen at two places:

1. When a relation lock waits behind an AccessExclusiveLock and then
Startup runs LockBufferForCleanup()

2. When Startup is a pin count waiter and a lock acquire begins to wait
on a relation lock

So we must put in direct deadlock detection in both places. We can't use
the normal deadlock detector because in case (1) the backend might
already have exceeded deadlock_timeout.

Proposal:

Make Startup wait on a well-known semaphore rather than on its
proc->sem. This means we can skip the spinlock check in
ProcSendSignal(). 

For (1) if Startup runs LockBufferForCleanup and can't get cleanup lock
then it marks itself waiting. It then checks for any lock waiters. If
there are >0 lock waiters then it waits for up to max_standby_delay and
then aborts all current lock waiters, none of whom would ever wake if we
continue waiting.

For (2) If a normal backend goes into a lock wait in HS then it will
check to see if Startup is waiting, if so, throw ERROR. This can happen
immediately because if Startup is already waiting then to wait for the
lock would cause deadlock.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range types

[tomas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Tue, Dec 15, 2009 at 11:49:19AM -0800, David Fetter wrote:
> On Tue, Dec 15, 2009 at 11:31:05AM -0800, Scott Bailey wrote:
> > Jeff Davis wrote:
> > >On Tue, 2009-12-15 at 10:19 -0500, Tom Lane wrote:
> > 
> > Would it be OK if we handled float timestamp ranges as continuous
> > and int64 timestamps discrete?
> 
> That sounds like a recipe for disaster.  Whatever timestamp ranges
> are, float and int64 should be treated the same way so as not to get
> "surprises" due to implementation details.

This alone would practically preclude discrete -- int and float would
behave quite differently (float's "granules" growing towards the edges
or having to choose a bigger granule for float than for int in the first
place).

[...]

> FWIW, I think it would be a good idea to treat timestamps as
> continuous in all cases.

This would come as a corollary from the above

Regards
- -- tomás
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFLKODXBcgs9XrR2kYRAlpLAJ9nO5f0SHwX8A4CjTn6c/xyZdim1ACdGHTq
Fwn5ygKvCDFGadufOYPGrfA=
=ivCP
-END PGP SIGNATURE-

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers