Re: [HACKERS] Updated tsearch documentation

[Oleg Bartunov]

On Wed, 18 Jul 2007, Bruce Momjian wrote:


Oleg, Teodor,

I am confused by the following example.  How does gin know to create a
tsvector, or does it?  Does gist know too?


No, gist doesn't know. I don't remember why, Teodor ?

For GIN see http://archives.postgresql.org/pgsql-hackers/2007-05/msg00625.php
for discussion



FYI, at some point we need to chat via instant messenger or IRC to
discuss the open items.  My chat information is here:

http://momjian.us/main/contact.html


I send you invitation for google talk, I use only chat in gmail.
My gmail account is [EMAIL PROTECTED]



---

SELECT title
FROM pgweb
WHERE textcat(title,body) @@ plainto_tsquery('create table')
ORDER BY dlm DESC LIMIT 10;

CREATE INDEX pgweb_idx ON pgweb USING gin(textcat(title,body));




Regards,
Oleg
_
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] Updated tsearch documentation

[Bruce Momjian]

Oleg, Teodor,

I am confused by the following example.  How does gin know to create a
tsvector, or does it?  Does gist know too?   

FYI, at some point we need to chat via instant messenger or IRC to
discuss the open items.  My chat information is here:

http://momjian.us/main/contact.html

---

SELECT title
FROM pgweb
WHERE textcat(title,body) @@ plainto_tsquery('create table')
ORDER BY dlm DESC LIMIT 10;

CREATE INDEX pgweb_idx ON pgweb USING gin(textcat(title,body));

-- 
  Bruce Momjian  <[EMAIL PROTECTED]>  http://momjian.us
  EnterpriseDB   http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org

Re: [HACKERS] Updated tsearch documentation

[Bruce Momjian]

Oleg Bartunov wrote:
> On Wed, 18 Jul 2007, Bruce Momjian wrote:
> 
> >
> >>> Why are we allowing my_filter_name here?  Isn't that something for a
> >>> custom trigger.  Is calling it tsearch() a good idea?  Why not
> >>> tsvector_trigger().
> >>
> >> I don't see any benefit from the tsvector_trigger() name. If you want to 
> >> add
> >> some semantic, than tsvector_update_trigger() would be better.  Anyway,
> >> this trigger is an illustration.
> >
> > Well, the filter that removes '@' might be an example, but tsearch() is
> > indeed sort of built-in trigger function to be used for simple cases.
> > My point is that because it is only for simple cases, why add complexity
> > and allow a filter?  It seems best to just remove the filter idea and
> > let people write their own triggers if they want that functionality.
> 
> If you aware about documentation simplicity than we could just document 
> two versions:
> 1. without filter function - simple, well understood syntax
> 2. with filter function - for advanced users
> 
> I don't want to remove the feature which works for year without any problem.

Yes, this is what I want.  I would like to show the simple usage first,
then explain that a more complex usage is possible.  This will help
people get started using text search.  Triggers and secondary columns
are fine, but to start using it the CREATE INDEX-only case is best.  I
don't suggest we remove any capabilities, only suggest simple solutions.

> > CREATE INDEX textsearch_id ON pgweb USING 
> > gin(to_tsvector(column));
> >
> > That avoids having to have a separate column because you can just say:
> >
> > WHERE to_query('XXX') @@ to_tsvector(column)
> 
>  yes, it's possible, but without ranking, since currently it's impossible
>  to store any information in index (it's pg's feature). btw, this should
>  works and for GiST index also.
> >>>
> >>> What if they use @@@.  Wouldn't that work because it is going to check
> >>> the heap?
> >>
> >> It would work, it'd recalculate to_tsvector(column) for rows found
> >> ( for GiST - to remove false hits and for weight information, for
> >> GIN - for weight information only).
> >
> > Right.  Currently to use text search on a table, you have to do three
> > things:
> >
> > o  add a tsvector column to the table
> > o  add a trigger to keep the tsvector column current
> > o  add an index to the tsvector column
> >
> > My question is why bother with the first two steps?  If you do:
> >
> > CREATE INDEX textsearch_idx ON pgweb USING 
> > gist(to_tsvector('english',column));
> >
> > you don't need a separate column and a trigger to keep it current.  The
> > index is kept current as part of normal query processing.  The only
> > downside is that you have to do to_tsvector() in the heap to avoid false
> > hits, but that seems minor compared to the disk savings of not having
> > the separate column.  Is to_tsvector() an expensive function?
> 
> Bruce, you oversimplify the text search, the document could be fully virtual,
> not a column(s), it could be a result of any SQL commands, so it could be 
> very expensive just to obtain document, and yes, to_tsvector could be
> very expensive, depending on the document size, parser and dictionaries used.
> 
> And, again, current postgres architecture forces to use heap to store
> positional and weight information for ranking.
> 
> The use case for what you described is very limited - simple text search
> on one/several column of the same table without ranking.

Right, but I bet that that is all the majority of users need, at least
at first as they start to use text search.

> >  CREATE INDEX textsearch_idx ON pgweb USING 
> > gin(to_tsvector('english',column));
> >
> > so that at least the configuration is documented in the index.
> 
>  yes, it's better to always explicitly specify configuration name and not
>  rely on default configuration.
>  Unfortunately, configuration name doesn't saved in the index.
> >>
> >> as Teodor corrected me, index doesn't know about configuration at all !
> >> What accurate user could do, is to provide configuration name in the
> >> comment for tsvector column. Configuration name is an accessory of
> >> to_tsvector() function.
> >
> > Well, if you create the index with the configuration name it is
> > guaranteed to match:
> >
> > CREATE INDEX textsearch_idx ON pgweb USING 
> > gist(to_tsvector('english',column));
> >  ---
> > And if someone does:
> >
> > WHERE 'friend'::tsquery @@ to_tsvector('english',column))
> >
> > the index is used.  Now if the default configuration is 'english' and
> > they use:
> >
> > WHERE 'friend'::tsquery @@ to_tsvector(column))
> >
> > the index is not used, but this just a good example of why default
> > configurations aren't that useful.  One problem I see is that if the
> > default configuration is not 

Re: [HACKERS] Why so many out-of-disk-space failures on buildfarm machines?

[Tom Lane]

Andrew Dunstan <[EMAIL PROTECTED]> writes:
> Tom Lane wrote:
>> BTW, while I'm thinking of it --- it'd be real nice if the buildfarm
>> "configuration" printout included the flex and bison version numbers.

> Interestingly, none of our tools actually outputs the bison/flex 
> versions - perhaps configure should be doing that.

Hmm, that's a good solution, especially since it'll start working right
away instead of waiting for buildfarm owners to update their scripts ;-)

I think I can get it to do
checking for flex... /usr/local/bin/flex 2.5.4
... does that seem reasonable?

> We do log the gcc 
> version (look in the config.log if you want to make sure).

Oh, that seems to be something in the base autoconf macros rather than
anything we put in.  I don't especially like hiding it in config.log
--- who reads that?

regards, tom lane

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly

Re: [HACKERS] Why so many out-of-disk-space failures on buildfarm machines?

[Andrew Dunstan]

Tom Lane wrote:

BTW, while I'm thinking of it --- it'd be real nice if the buildfarm
"configuration" printout included the flex and bison version numbers.
Maybe gcc too (I know not every buildfarm member is compiling with gcc,
but it comes in enough different versions that this is likely to be
useful info).


  


Interestingly, none of our tools actually outputs the bison/flex 
versions - perhaps configure should be doing that. We do log the gcc 
version (look in the config.log if you want to make sure).


cheers

andrew


---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

   http://www.postgresql.org/about/donate

Re: [HACKERS] Why so many out-of-disk-space failures on buildfarm machines?

[Tom Lane]

BTW, while I'm thinking of it --- it'd be real nice if the buildfarm
"configuration" printout included the flex and bison version numbers.
Maybe gcc too (I know not every buildfarm member is compiling with gcc,
but it comes in enough different versions that this is likely to be
useful info).

regards, tom lane

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org

Re: [HACKERS] Why so many out-of-disk-space failures on buildfarm machines?

[Tom Lane]

Andrew Dunstan <[EMAIL PROTECTED]> writes:
> I don't think we're ever going to fix things for the 7.3 error you're 
> getting - please take it out of your rotation. 7.3 isn't quite as dead 
> as Joshua suggested earlier, but it's certainly on life support.

I checked the CVS logs and it appears that we fixed several contrib
modules, not only cube, to work with flex 2.5.31 during the 7.4 devel
cycle.  I don't think anyone cares to back-port that much work.  Our
position should be "if you want to build 7.3 you need flex 2.5.4 to do
it".

If Mark still wants to test 7.3, he could install flex 2.5.4 someplace
and make sure that's first in the PATH while building 7.3.

regards, tom lane

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match

Re: [HACKERS] Why so many out-of-disk-space failures on buildfarm machines?

[Andrew Dunstan]

Mark Wong wrote:

On 7/3/07, Tom Lane <[EMAIL PROTECTED]> wrote:

wombat  long-standing configuration error (no Tk installed)


My apologies for not responding earlier.  I see 7.3 contrib problems
for wombat but I don't see a config error for Tk with HEAD or any of
the other 8.x releases.  I have the --without-tk flag for the 7.4
release.




Mark,


I don't think we're ever going to fix things for the 7.3 error you're 
getting - please take it out of your rotation. 7.3 isn't quite as dead 
as Joshua suggested earlier, but it's certainly on life support.


cheers

andrew

---(end of broadcast)---
TIP 6: explain analyze is your friend

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Tom Lane ([EMAIL PROTECTED]) wrote:
> Oh, they're fully interchangeable at the wire level?  Is this true both
> with respect to the PG client/backend protocol and the protocol to the
> authentication server?

I believe that's the case, yes.

> If there's no interoperability issues then I
> agree that a configure-time choice is sufficient for selecting which
> library to use.

In general I agree, but I'd like to see builds for Windows which support
them and I'm not sure that'll happen quite as regularly. :/

Aside from that issue though, if we're going to continue krb5 support
(which I'd encourage unless we have some reason to stop) and it's not
too much effort (I get the impression it's not) to support both
concurrently, I'd really appreciate it. :)  I'm not aware of any 'funny
business' which would be involved in supporting them both at the same
time, and I believe Magnus is working on it.

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Gregory Stark ([EMAIL PROTECTED]) wrote:
> Am I right in thinking that while the client<->postgres protocol may be the
> same the actual authentication tokens are different? That is, if you have a
> Windows Active Directory server then using SSPI will use your Windows
> credentials obtained from that server to log you in whereas if you used the
> MIT GSSAPI library it would try to use your Kerberos tickets for which it 
> would
> look elsewhere?

This *can* be true, and in fact is *exactly* what I do.  The MIT client
comes with an option (enabled by default actually) to sync up the MIT
ticket cache with the SSPI one though.

> What confuses me here is that I don't understand how this relates to
> applications. You keep talking about using the connection string which may be
> appropriate for a user-oriented application like psql. But in the general case
> surely the application needs to be able to control the authentication process
> and be able to provide credentials of its choice?

We're talking about user-oriented applications...  Specifically things
like psql and Postgres ODBC, which use user's credentials to connect to
the database and don't have their own credentials...

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Why so many out-of-disk-space failures on buildfarm machines?

[Tom Lane]

"Mark Wong" <[EMAIL PROTECTED]> writes:
> On 7/3/07, Tom Lane <[EMAIL PROTECTED]> wrote:
>> wombat  long-standing configuration error (no Tk installed)

> My apologies for not responding earlier.  I see 7.3 contrib problems
> for wombat but I don't see a config error for Tk with HEAD or any of
> the other 8.x releases.  I have the --without-tk flag for the 7.4
> release.

Sorry about that, I must've confused wombat with some other buildfarm
critter while making that list.

On inspection it looks like a flex compatibility issue.  What version of
flex is on that machine?

regards, tom lane

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq

Re: [HACKERS] Future of krb5 authentication

[Tom Lane]

Magnus Hagander <[EMAIL PROTECTED]> writes:
> The issue is *not* about GSSAPI vs krb5. It's with GSSAPI vs SSPI.

> The wire protocol is the same for them. It's a matter of which *client
> library* should be used to produce the packets that go over the network.

Oh, they're fully interchangeable at the wire level?  Is this true both
with respect to the PG client/backend protocol and the protocol to the
authentication server?  If there's no interoperability issues then I
agree that a configure-time choice is sufficient for selecting which
library to use.

regards, tom lane

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Re: [HACKERS] Future of krb5 authentication

[Gregory Stark]

"Heikki Linnakangas" <[EMAIL PROTECTED]> writes:

> Magnus Hagander wrote:
>
>> The wire protocol is the same for them. It's a matter of which *client
>> library* should be used to produce the packets that go over the network.
>...
> On Windows, why would you need GSSAPI, if SSPI comes with the operation
> system? What's the difference between the libraries? Can you try SSPI
> first, and fall back to GSSAPI?

Am I right in thinking that while the client<->postgres protocol may be the
same the actual authentication tokens are different? That is, if you have a
Windows Active Directory server then using SSPI will use your Windows
credentials obtained from that server to log you in whereas if you used the
MIT GSSAPI library it would try to use your Kerberos tickets for which it would
look elsewhere?

What confuses me here is that I don't understand how this relates to
applications. You keep talking about using the connection string which may be
appropriate for a user-oriented application like psql. But in the general case
surely the application needs to be able to control the authentication process
and be able to provide credentials of its choice?

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com


---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Heikki Linnakangas ([EMAIL PROTECTED]) wrote:
> Uh, this is really confusing. Let's see if I got this right. So we're
> talking about two orthogonal changes here:

It is kinda confusing. :)

> 1. Wire protocol. In 8.2 and below, we used the krb5 protocol. 8.3
> server and libpq will use the GSSAPI wire protocol by default, with
> support for krb5 protocol when speaking with older versions.

Well, I think it'll depend on what's configured, no?  Doesn't the libpq
protocol say back to the user "this is what I want to use" or similar?
The impression I got was more along the lines of- we'll have another
option in pg_hba.conf for 'gssapi', distinct from 'krb5' and either
could be used.  Might have misunderstood tho.

> 2. In 8.2 and below, we used the GSSAPI library on all platforms. 8.3
> adds support for Microsoft's SSPI interface on Windows.

No..  We used the MIT Krb5 library.  This is a change to use the GSSAPI
library (also from MIT and part of their Kerberos distribution, so it's
a tad confusing) on Unix by default and compile in support for it under
Windows as well.

> On Windows, why would you need GSSAPI, if SSPI comes with the operation
> system? What's the difference between the libraries? Can you try SSPI
> first, and fall back to GSSAPI?

You can't really 'fall back' without creating alot of noise in the logs
and whatnot.  Also, it could try to do things that don't make any sense.
The reason to support both is that they have, essentially, different
feature sets.

> Can you do <= 8.2 style krb5 authentication with the SSPI library?

No, at least from a user-interface standpoint and I think also the
wireline protocol is different...

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Why so many out-of-disk-space failures on buildfarm machines?

[Mark Wong]

On 7/3/07, Tom Lane <[EMAIL PROTECTED]> wrote:

wombat  long-standing configuration error (no Tk installed)


My apologies for not responding earlier.  I see 7.3 contrib problems
for wombat but I don't see a config error for Tk with HEAD or any of
the other 8.x releases.  I have the --without-tk flag for the 7.4
release.

Regards,
Mark

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] Future of krb5 authentication

[Heikki Linnakangas]

Magnus Hagander wrote:
> Heikki Linnakangas wrote:
>> Stephen Frost wrote:
>>> Honestly, for now I'm happy w/ it being a connectionstring option.  It
>>> seems the most appropriate place for it to go.  That does mean that
>>> applications may need to be modified to support gssapi (where they might
>>> not have to be for sspi since it's the default), but since we're going
>>> to keep krb5 support around for a bit there's time for those
>>> applications to catch up without breaking things explicitly for people
>>> migrating to 8.3.
>> Isn't it possible to open the socket, try GSSAPI handshaking with
>> protocol, and fall back to krb5 protocol if that fails? If that's not
>> possible, how about handling it like we handle postgres protocol 3 vs 2?
>> Connect using GSSAPI first, and if that fails, retry with krb5.
> 
> The issue is *not* about GSSAPI vs krb5. It's with GSSAPI vs SSPI.
> 
> The wire protocol is the same for them. It's a matter of which *client
> library* should be used to produce the packets that go over the network.

Uh, this is really confusing. Let's see if I got this right. So we're
talking about two orthogonal changes here:

1. Wire protocol. In 8.2 and below, we used the krb5 protocol. 8.3
server and libpq will use the GSSAPI wire protocol by default, with
support for krb5 protocol when speaking with older versions.

2. In 8.2 and below, we used the GSSAPI library on all platforms. 8.3
adds support for Microsoft's SSPI interface on Windows.

On Windows, why would you need GSSAPI, if SSPI comes with the operation
system? What's the difference between the libraries? Can you try SSPI
first, and fall back to GSSAPI?

Can you do <= 8.2 style krb5 authentication with the SSPI library?

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

Heikki Linnakangas wrote:
> Stephen Frost wrote:
>> Honestly, for now I'm happy w/ it being a connectionstring option.  It
>> seems the most appropriate place for it to go.  That does mean that
>> applications may need to be modified to support gssapi (where they might
>> not have to be for sspi since it's the default), but since we're going
>> to keep krb5 support around for a bit there's time for those
>> applications to catch up without breaking things explicitly for people
>> migrating to 8.3.
> 
> Isn't it possible to open the socket, try GSSAPI handshaking with
> protocol, and fall back to krb5 protocol if that fails? If that's not
> possible, how about handling it like we handle postgres protocol 3 vs 2?
> Connect using GSSAPI first, and if that fails, retry with krb5.

The issue is *not* about GSSAPI vs krb5. It's with GSSAPI vs SSPI.

The wire protocol is the same for them. It's a matter of which *client
library* should be used to produce the packets that go over the network.

//Magnus


---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org

Re: [HACKERS] Updated tsearch documentation

[Oleg Bartunov]

On Wed, 18 Jul 2007, Bruce Momjian wrote:




Why are we allowing my_filter_name here?  Isn't that something for a
custom trigger.  Is calling it tsearch() a good idea?  Why not
tsvector_trigger().


I don't see any benefit from the tsvector_trigger() name. If you want to add
some semantic, than tsvector_update_trigger() would be better.  Anyway,
this trigger is an illustration.


Well, the filter that removes '@' might be an example, but tsearch() is
indeed sort of built-in trigger function to be used for simple cases.
My point is that because it is only for simple cases, why add complexity
and allow a filter?  It seems best to just remove the filter idea and
let people write their own triggers if they want that functionality.


If you aware about documentation simplicity than we could just document 
two versions:

1. without filter function - simple, well understood syntax
2. with filter function - for advanced users

I don't want to remove the feature which works for year without any problem.





CREATE INDEX textsearch_id ON pgweb USING gin(to_tsvector(column));

That avoids having to have a separate column because you can just say:

WHERE to_query('XXX') @@ to_tsvector(column)


yes, it's possible, but without ranking, since currently it's impossible
to store any information in index (it's pg's feature). btw, this should
works and for GiST index also.


What if they use @@@.  Wouldn't that work because it is going to check
the heap?


It would work, it'd recalculate to_tsvector(column) for rows found
( for GiST - to remove false hits and for weight information, for
GIN - for weight information only).


Right.  Currently to use text search on a table, you have to do three
things:

o  add a tsvector column to the table
o  add a trigger to keep the tsvector column current
o  add an index to the tsvector column

My question is why bother with the first two steps?  If you do:

CREATE INDEX textsearch_idx ON pgweb USING gist(to_tsvector('english',column));

you don't need a separate column and a trigger to keep it current.  The
index is kept current as part of normal query processing.  The only
downside is that you have to do to_tsvector() in the heap to avoid false
hits, but that seems minor compared to the disk savings of not having
the separate column.  Is to_tsvector() an expensive function?


Bruce, you oversimplify the text search, the document could be fully virtual,
not a column(s), it could be a result of any SQL commands, so it could be 
very expensive just to obtain document, and yes, to_tsvector could be

very expensive, depending on the document size, parser and dictionaries used.

And, again, current postgres architecture forces to use heap to store
positional and weight information for ranking.

The use case for what you described is very limited - simple text search
on one/several column of the same table without ranking.




 CREATE INDEX textsearch_idx ON pgweb USING gin(to_tsvector('english',column));

so that at least the configuration is documented in the index.


yes, it's better to always explicitly specify configuration name and not
rely on default configuration.
Unfortunately, configuration name doesn't saved in the index.


as Teodor corrected me, index doesn't know about configuration at all !
What accurate user could do, is to provide configuration name in the
comment for tsvector column. Configuration name is an accessory of
to_tsvector() function.


Well, if you create the index with the configuration name it is
guaranteed to match:

CREATE INDEX textsearch_idx ON pgweb USING gist(to_tsvector('english',column));
 ---
And if someone does:

WHERE 'friend'::tsquery @@ to_tsvector('english',column))

the index is used.  Now if the default configuration is 'english' and
they use:

WHERE 'friend'::tsquery @@ to_tsvector(column))

the index is not used, but this just a good example of why default
configurations aren't that useful.  One problem I see is that if the
default configuration is not 'english', then when the index consults the
heap, it would be using a different configuration and yield incorrect
results.  I am unsure how to fix that.


again, you consider very simple case  and actually, your example is a 
good example of usefulness of default configuration ! Just think before

you develop your application, but this is very general rule. There are
zillions situations you could do bad things, after all.

Moreover, consider text search on text column, there is no way to specify 
configuration at all ! We rely on default configuration here


CREATE INDEX textsearch_idx ON pgweb USING gin(title);



With the trigger idea, you have to be sure your configuration is the same
every time you INSERT/UPDATE the table or the index will have mixed
configuration entries and it will yield incorrect results, aside from
the heap configuration lookup not matching the index.

Re: [HACKERS] Future of krb5 authentication

[Heikki Linnakangas]

Stephen Frost wrote:
> Honestly, for now I'm happy w/ it being a connectionstring option.  It
> seems the most appropriate place for it to go.  That does mean that
> applications may need to be modified to support gssapi (where they might
> not have to be for sspi since it's the default), but since we're going
> to keep krb5 support around for a bit there's time for those
> applications to catch up without breaking things explicitly for people
> migrating to 8.3.

Isn't it possible to open the socket, try GSSAPI handshaking with
protocol, and fall back to krb5 protocol if that fails? If that's not
possible, how about handling it like we handle postgres protocol 3 vs 2?
Connect using GSSAPI first, and if that fails, retry with krb5.


-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Magnus Hagander ([EMAIL PROTECTED]) wrote:
> Well, since you're the only one who've asked for the feature, I guess
> that's good enough for me unless someone else complains. If you have a
> good suggestion for a name for it, let me know, otherwise I'll just cook
> something up.

Mozilla uses 'gsslib', that would work for me.

ie:

gsslib=gss
gsslib=sspi

Also, thanks!

> BTW, I have working SSPI server code running on Windows now as well,
> giving the full Active Directory integration in the windows postgresql.
> It's far from perfect yet, needs a bunch of cleanup, but it works. (This
> is SSPI, of course)

Awesome!  Very exciting. :)

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

Stephen Frost wrote:
> * Magnus Hagander ([EMAIL PROTECTED]) wrote:
>> Stephen Frost wrote:
>>> * Magnus Hagander ([EMAIL PROTECTED]) wrote:
 Certainly not "just minor adjustments", since we need to do dynamic
 loading and checking for the functions. That's the big one, which will
>>> If we're supporting krb5 anyway, and shipping the bits that go along
>>> with that, do we need to do dynamic loading and function checking?
>> Eh, good point. I got confused, it seems :-) Scratch that, then - we're
>> back to finding a good way to specify it.
> 
> Honestly, for now I'm happy w/ it being a connectionstring option.  It
> seems the most appropriate place for it to go.  That does mean that
> applications may need to be modified to support gssapi (where they might
> not have to be for sspi since it's the default), but since we're going
> to keep krb5 support around for a bit there's time for those
> applications to catch up without breaking things explicitly for people
> migrating to 8.3.

Well, since you're the only one who've asked for the feature, I guess
that's good enough for me unless someone else complains. If you have a
good suggestion for a name for it, let me know, otherwise I'll just cook
something up.

BTW, I have working SSPI server code running on Windows now as well,
giving the full Active Directory integration in the windows postgresql.
It's far from perfect yet, needs a bunch of cleanup, but it works. (This
is SSPI, of course)

//Magnus


---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] Future of krb5 authentication

[Joshua D. Drake]

Stephen Frost wrote:

* Joshua D. Drake ([EMAIL PROTECTED]) wrote:



Oh, yea, and every place that uses Active Directory ..

Note that we are talking about Kerberos + PostgreSQL, not Kerberose in 
general.


I was referring to your first question, which, in my view, is the more
appropriate one *anyway*.  At least if you have a "PostgreSQL is going
to dominate the WORLD!" point of view, as I do. :) 


Of course I do. I am just a realist and know that isn't going to happen 
on Win32 ;)


Joshua D. Drake



Thanks,

Stephen



--

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997
 http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/


---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Joshua D. Drake ([EMAIL PROTECTED]) wrote:
> Stephen Frost wrote:
>> * Joshua D. Drake ([EMAIL PROTECTED]) wrote:
>>> How many people actually use kerberos... How many people who are using 
>>> kerberos are going to be running 7.3. 7.3 is no longer supported so by 
>>> postgresql.org so who cares.
>> AOL, MIT, CMU, to name a few...  I'm really annoyed at these constant
>> digs at what is really a very large userbase.  Perhaps they're not all
>> running 7.3 but the implication that there's a small number of people
>> using Kerberos is just amazingly far off.
>
> *cough* , compared to the number of installations *not* using kerberos it 
> is amazing that you would make such a far off correlation.

Oh, yea, and every place that uses Active Directory ..

> Note that we are talking about Kerberos + PostgreSQL, not Kerberose in 
> general.

I was referring to your first question, which, in my view, is the more
appropriate one *anyway*.  At least if you have a "PostgreSQL is going
to dominate the WORLD!" point of view, as I do. :) 

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Magnus Hagander ([EMAIL PROTECTED]) wrote:
> Stephen Frost wrote:
> > * Magnus Hagander ([EMAIL PROTECTED]) wrote:
> >> Certainly not "just minor adjustments", since we need to do dynamic
> >> loading and checking for the functions. That's the big one, which will
> > 
> > If we're supporting krb5 anyway, and shipping the bits that go along
> > with that, do we need to do dynamic loading and function checking?
> 
> Eh, good point. I got confused, it seems :-) Scratch that, then - we're
> back to finding a good way to specify it.

Honestly, for now I'm happy w/ it being a connectionstring option.  It
seems the most appropriate place for it to go.  That does mean that
applications may need to be modified to support gssapi (where they might
not have to be for sspi since it's the default), but since we're going
to keep krb5 support around for a bit there's time for those
applications to catch up without breaking things explicitly for people
migrating to 8.3.

I'd also use that as an opportunity to encourage applications to expose
the connectionstring to users as there may be things like this in the
future where it's purely a library thing and the application doesn't
have to know about it- except for the connectionstring.

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Magnus Hagander ([EMAIL PROTECTED]) wrote:
> No, no requirement. But you would certainly expect it to use it if you
> have SSL on the connection.

Uhh, perhaps, but my recollection is that it's generally *not* done that
way in other things..  Honestly, it doesn't matter to me, just wanted to
clear up the implication.

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Updated tsearch documentation

[Bruce Momjian]

Oleg Bartunov wrote:
> >> I agree, that there are could be more examples, but text search doesn't
> >> require something special !
> >> *Example* of trigger function is documented on
> >> http://momjian.us/expire/fulltext/HTML/textsearch-opfunc.html
> >
> > Yes, I see that in tsearch() here:
> >
> > http://momjian.us/expire/fulltext/HTML/textsearch-opfunc.html#TEXTSEARC$
> >
> > I assume my_filter_name is optional right?  I have updated the prototype
> > to be:
> >
> > tsearch([vector_column_name], [my_filter_name], text_column_name [, ... 
> > ])
> >
> > Is this accurate?  What does this text below it mean?
> 
> no, this in inaccurate. First, vector_column_name is not optional argument,
> it's a name of tsvector column name.

Fixed.

> > There can be many functions and text columns specified in a tsearch()
> > trigger. The following rule is used: a function is applied to all
> > subsequent TEXT columns until the next matching column occurs.
> 
> The idea, is to provide user to preprocess text before applying 
> tsearch machinery. my_filter_name() preprocess text_column_name1,
> text_column_name2,
> The original syntax allows to specify for every text columns their 
> preprocessing functions.
> 
> So, I suggest to keep original syntax, change 'vector_column_name' to
> 'tsvector_column_name'.

OK, change made.

> > Why are we allowing my_filter_name here?  Isn't that something for a
> > custom trigger.  Is calling it tsearch() a good idea?  Why not
> > tsvector_trigger().
> 
> I don't see any benefit from the tsvector_trigger() name. If you want to add
> some semantic, than tsvector_update_trigger() would be better.  Anyway,
> this trigger is an illustration.

Well, the filter that removes '@' might be an example, but tsearch() is
indeed sort of built-in trigger function to be used for simple cases. 
My point is that because it is only for simple cases, why add complexity
and allow a filter?  It seems best to just remove the filter idea and
let people write their own triggers if they want that functionality.

> >>>   CREATE INDEX textsearch_id ON pgweb USING gin(to_tsvector(column));
> >>>
> >>> That avoids having to have a separate column because you can just say:
> >>>
> >>>   WHERE to_query('XXX') @@ to_tsvector(column)
> >>
> >> yes, it's possible, but without ranking, since currently it's impossible
> >> to store any information in index (it's pg's feature). btw, this should
> >> works and for GiST index also.
> >
> > What if they use @@@.  Wouldn't that work because it is going to check
> > the heap?
> 
> It would work, it'd recalculate to_tsvector(column) for rows found
> ( for GiST - to remove false hits and for weight information, for 
> GIN - for weight information only).

Right.  Currently to use text search on a table, you have to do three
things:

o  add a tsvector column to the table
o  add a trigger to keep the tsvector column current
o  add an index to the tsvector column

My question is why bother with the first two steps?  If you do:

 CREATE INDEX textsearch_idx ON pgweb USING gist(to_tsvector('english',column));

you don't need a separate column and a trigger to keep it current.  The
index is kept current as part of normal query processing.  The only
downside is that you have to do to_tsvector() in the heap to avoid false
hits, but that seems minor compared to the disk savings of not having
the separate column.  Is to_tsvector() an expensive function?

> >>>  CREATE INDEX textsearch_idx ON pgweb USING 
> >>> gin(to_tsvector('english',column));
> >>>
> >>> so that at least the configuration is documented in the index.
> >>
> >> yes, it's better to always explicitly specify configuration name and not
> >> rely on default configuration.
> >> Unfortunately, configuration name doesn't saved in the index.
> 
> as Teodor corrected me, index doesn't know about configuration at all !
> What accurate user could do, is to provide configuration name in the 
> comment for tsvector column. Configuration name is an accessory of
> to_tsvector() function.

Well, if you create the index with the configuration name it is
guaranteed to match:

 CREATE INDEX textsearch_idx ON pgweb USING gist(to_tsvector('english',column));
  ---
And if someone does:

WHERE 'friend'::tsquery @@ to_tsvector('english',column))

the index is used.  Now if the default configuration is 'english' and
they use:

WHERE 'friend'::tsquery @@ to_tsvector(column))

the index is not used, but this just a good example of why default
configurations aren't that useful.  One problem I see is that if the
default configuration is not 'english', then when the index consults the
heap, it would be using a different configuration and yield incorrect
results.  I am unsure how to fix that.

With the trigger idea, you have to be sure your configuration is the same
every time you INSERT/UPDATE the table or the index will have

Re: [HACKERS] Comments on the HOT design

[Pavan Deolasee]

On 7/18/07, Heikki Linnakangas <[EMAIL PROTECTED] > wrote:


Here's what I think we should do to the HOT patch:




I am all for simplifying the code. That would not only help us make it less
buggy but also improve its maintainability. But we would also need
to repeat the tests and run new tests to make sure that the simplification
does not come at a significant loss of performance gain.


1. Get rid of row-level fragmentation and handling dealing with

LP_DELETEd line pointers. Instead, take a vacuum lock opportunistically,
and defrag pages using the normal PageRepairFragmentation function. I'm
not sure where exactly we would do the pruning and where we would call
PageRepairFragmentation. We could do it in heap_release_fetch, but we
need some logic to decide when it's helpful and when it's a waste of time.




I think its worth trying for this simplification. Though I am not sure
if this is the most complicated part of the code. Nevertheless, lesser
the complexity, better it is.

One thing that strikes me is that we are assuming HOT to benefit
large tables. This is mostly true, but I am wondering how this
simplification would impact the HOT effect on small tables.

I assume you are suggesting *not* to call PageRepairFragmentation
in heap_update given that it holds reference to the old tuple.


2. Get rid of separate handling and WAL record types for pruning aborted

tuples, ISTM that's no longer needed with the simplified pruning method.




Ok. If we can do that, its well and good.


3. Currently, the patch has a separate code path for pruning pages in

VACUUM FULL, which removes any DEAD tuples in the middle of chains, and
fixes the ctid/xmin of the previous/next tuple to keep the chain valid.
Instead, we should just prune all the preceding tuples in the chain,
since they're in fact dead as well. Our simplistic check with OldestXmin
just isn't enough to notice that. I think we can share most of the code
between normal pruning and VACUUM FULL, which is good because the VACUUM
FULL codepath is used very seldom, so if there's any bugs in there they
might go unnoticed for a long time.




Yes. We should certainly do that. We have discussed this before on the
list and we had concluded that any tuples preceding a DEAD tuple are
also definitely DEAD.


4. Write only one WAL record per pruned page, instead of one per update

chain.




Ok. Currently the page pruning just prunes all the individual chains
in the page. So the code just reuses the WAL logging for per chain
pruning (which we anyways need for stand-alone chain pruning).
Are you suggesting we should move away from chain pruning during
index fetch ?


I've done some experimenting on those items, producing several badly

broken versions of the patch partly implementing those ideas. It looks
like the patch size will go down from ~240 kB to ~210 kB, and more
importantly, there will be less new concepts and states a tuple can be
in and less WAL record types.




Thats very good. I would appreciate if you continue the refactoring
process. We can coordinate so that our work doesn't conflict.


I know we've been running DBT-2 tests with the patch, and that it's

effective in reducing the need to vacuum the big tables which gives
better throughput in long runs. But I also know that a lot of people are
interested in the potential to avoid CPU overhead of index inserts. We
need to run CPU bound benchmarks to measure that effect as well.




Sure. May be we should also measure effects on small/large tables,
different mix of HOT and COLD updates etc.


Thanks,
Pavan


--
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] Future of krb5 authentication

[Joshua D. Drake]

Stephen Frost wrote:

* Joshua D. Drake ([EMAIL PROTECTED]) wrote:

OK, well thats a problem. pgAdmin supports back to 7.3...
How many people actually use kerberos... How many people who are using 
kerberos are going to be running 7.3. 7.3 is no longer supported so by 
postgresql.org so who cares.


AOL, MIT, CMU, to name a few...  I'm really annoyed at these constant
digs at what is really a very large userbase.  Perhaps they're not all
running 7.3 but the implication that there's a small number of people
using Kerberos is just amazingly far off.


*cough* , compared to the number of installations *not* using kerberos 
it is amazing that you would make such a far off correlation.


Note that we are talking about Kerberos + PostgreSQL, not Kerberose in 
general.


Joshua D. Drake




Thanks,

Stephen



--

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997
 http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/


---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

Stephen Frost wrote:
> * Magnus Hagander ([EMAIL PROTECTED]) wrote:
>> Certainly not "just minor adjustments", since we need to do dynamic
>> loading and checking for the functions. That's the big one, which will
> 
> If we're supporting krb5 anyway, and shipping the bits that go along
> with that, do we need to do dynamic loading and function checking?

Eh, good point. I got confused, it seems :-) Scratch that, then - we're
back to finding a good way to specify it.

//Magnus


---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

Stephen Frost wrote:
> * Magnus Hagander ([EMAIL PROTECTED]) wrote:
>> The maintenance part of me suggesting getting rid of krb5 is the
>> smallest one. It being a non-standard protocol is more important, and
>> the fact that the exchange breaks the libpq protocol and is not
>> protected by SSL is the big reason.
> 
> Erm, it doesn't need to be protected by SSL?  Breaking the libpq
> protocol does kind of suck.  I assume you're not requiring SSL for the
> GSSAPI stuff...

No, no requirement. But you would certainly expect it to use it if you
have SSL on the connection.

//Magnus


---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] SSPI authentication

[plabrh1]

Just the SSPI piece.  

Right now we run a mixture of PostgreSQL and SQL Server and the one
fustrating thing is that we have to have separate security architectures for
them.  The SQL Server environment is nice because it allows SSPI and
eliminates the need to pass around passwords everywhere.  

In the postgres environment, we've worked around that by "Trusting" the
postgres user from certain locked down and protected IP addresses so that we
don't need to store passwords but that would never win any security awards.
:)

SSPI will enable us to create services that run as that registered user and
as long as that user can obtain an authenticated kerb ticket, we can ensure
that are the user they say they are.  Much nicer model...

Looking forward to this release.  When will it be available?

Paul







-Original Message-
From: Magnus Hagander [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, July 18, 2007 6:42 AM
To: Paul Silveira
Cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] SSPI authentication

On Tue, Jul 17, 2007 at 11:00:35AM -0700, Paul Silveira wrote:
> 
> This is great.  I've worked on 2 projects in the last year that
desperately
> needed this.  It will certainly make the security model more seamless...

Thanks for letting us know.

Are you interested in just the SSPI parts, or also in being able to use
both SSPI and GSSAPI at the same time?

//Magnus


---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Magnus Hagander ([EMAIL PROTECTED]) wrote:
> Certainly not "just minor adjustments", since we need to do dynamic
> loading and checking for the functions. That's the big one, which will

If we're supporting krb5 anyway, and shipping the bits that go along
with that, do we need to do dynamic loading and function checking?

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Magnus Hagander ([EMAIL PROTECTED]) wrote:
> The maintenance part of me suggesting getting rid of krb5 is the
> smallest one. It being a non-standard protocol is more important, and
> the fact that the exchange breaks the libpq protocol and is not
> protected by SSL is the big reason.

Erm, it doesn't need to be protected by SSL?  Breaking the libpq
protocol does kind of suck.  I assume you're not requiring SSL for the
GSSAPI stuff...

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

Stephen Frost wrote:
> * Tom Lane ([EMAIL PROTECTED]) wrote:
>> Magnus Hagander <[EMAIL PROTECTED]> writes:
>>> On Wed, Jul 18, 2007 at 10:46:58AM -0400, Tom Lane wrote:
 This needs to be fixed.
>>> Non, GSSAPI and krb5 are *not* mutually exclusive.
>>> SSPI and GSSAPI are mutually exclusive.
>> Color me confused then.  What's the difference?
> 
> GSSAPI is the MIT libraries, SSPI is the Windows library, but there's no
> way to indicate to libpq which to use and they share some of the same
> code paths with minor adjustments for each done at compile-time (aiui
> anyway, Magnus can provide a clearer answer on this).
> 

Certainly not "just minor adjustments", since we need to do dynamic
loading and checking for the functions. That's the big one, which will
certainly increase the required code a lot. The part about letting the
client specify how is probably fairly easy, if we can figure out a good
one. (I personally think we've clearly shown that using the
connectionstring is not a good enough way to do it)

//Magnus


---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

Stephen Frost wrote:
> * Magnus Hagander ([EMAIL PROTECTED]) wrote:
>> But we're talking two different issues. Deprecating/removing krb5 is a
>> different thing from having GSSAPI and SSPI mutually exclusive or not.
> 
> To the extent that keeping krb5 around implies a much lower burden on
> GSSAPI support under Windows, I disagree...  If we need the MIT
> headers/libraries around to support krb5 anyway then I don't feel the
> fact that you can do SSPI w/o those headers/libraries to be a case for
> not supporting GSSAPI on Windows, we need them anyway...

I was talking from a technical perspective, not a maintenance one.

Your argument is at least party valid - though Dave has reported major
issues due to the gssapi library changing between versions. But those
are solvable.

The maintenance part of me suggesting getting rid of krb5 is the
smallest one. It being a non-standard protocol is more important, and
the fact that the exchange breaks the libpq protocol and is not
protected by SSL is the big reason.

But none of those more important reasons speak for removing krb5 - just
deprecating it. So I'm fine with doing that.

(and again, the SSPI vs GSSAPI on win32 discussion is a different one)

//Magnus

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Joshua D. Drake ([EMAIL PROTECTED]) wrote:
>> OK, well thats a problem. pgAdmin supports back to 7.3...
>
> How many people actually use kerberos... How many people who are using 
> kerberos are going to be running 7.3. 7.3 is no longer supported so by 
> postgresql.org so who cares.

AOL, MIT, CMU, to name a few...  I'm really annoyed at these constant
digs at what is really a very large userbase.  Perhaps they're not all
running 7.3 but the implication that there's a small number of people
using Kerberos is just amazingly far off.

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Dave Page ([EMAIL PROTECTED]) wrote:
> Probably not in the majority of cases - but we have a large userbase these 
> days, and a small percentage may still equate to a large number. I know at 
> least two people that do use psqlODBC + Kerberos.

I certainly use it alot!  Of course, we'll move to GSSAPI, assuming
that's not too difficult for us to do under Windows...

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Magnus Hagander ([EMAIL PROTECTED]) wrote:
> But we're talking two different issues. Deprecating/removing krb5 is a
> different thing from having GSSAPI and SSPI mutually exclusive or not.

To the extent that keeping krb5 around implies a much lower burden on
GSSAPI support under Windows, I disagree...  If we need the MIT
headers/libraries around to support krb5 anyway then I don't feel the
fact that you can do SSPI w/o those headers/libraries to be a case for
not supporting GSSAPI on Windows, we need them anyway...

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Future of krb5 authentication

[Stephen Frost]

* Tom Lane ([EMAIL PROTECTED]) wrote:
> Magnus Hagander <[EMAIL PROTECTED]> writes:
> > On Wed, Jul 18, 2007 at 10:46:58AM -0400, Tom Lane wrote:
> >> This needs to be fixed.
> 
> > Non, GSSAPI and krb5 are *not* mutually exclusive.
> 
> > SSPI and GSSAPI are mutually exclusive.
> 
> Color me confused then.  What's the difference?

GSSAPI is the MIT libraries, SSPI is the Windows library, but there's no
way to indicate to libpq which to use and they share some of the same
code paths with minor adjustments for each done at compile-time (aiui
anyway, Magnus can provide a clearer answer on this).

The feeling was that because there's no way to indicate to libpq which
to use except through the connectionstring and that most people would
want SSPI instead and that krb5 support is going to be removed that we
could just support either SSPI or GSSAPI (not both).

My feeling is that if we're going to continue to support krb5 *anyway*
(which I don't disagree with, honestly) then the GSSAPI stuff is going
to be required for the build *regardless* and therefore it makes sense
to support both in libpq rather than making them mutually exclusive.
Supporting it using a connectionstring option would be sufficient, imv,
though downstream utilities that don't let you modify the
connectionstring directly would have to add support for it (I'm of the
opinion that such things should be changed to allow a connectionstring
option, or at least an append to it, but perhaps there's some reason
that's a problem for some).

GSSAPI and SSPI are not, themselves, mutually exclusive.  They're just
being made that way by the libpq code that's been proposed.  Mozilla
handles doing both just fine and you flip between them using an option
in their 'about:config' screen.

Thanks,

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

Tom Lane wrote:
> Magnus Hagander <[EMAIL PROTECTED]> writes:
>> On Wed, Jul 18, 2007 at 10:46:58AM -0400, Tom Lane wrote:
>>> This needs to be fixed.
> 
>> Non, GSSAPI and krb5 are *not* mutually exclusive.
> 
>> SSPI and GSSAPI are mutually exclusive.
> 
> Color me confused then.  What's the difference?

SSPI is a Windows-only implementation of the GSSAPI protocol, that has a
different API.

GSSAPI works on Unix and on Windows (but only with addon libraries, such
as MIT (unix or win) or Heimdal (unix only)).

The confusion probably comes from that GSSAPI is both a protocol
(supported by SSPI as well) and an API (not supported by SSPI).

Now, SSPI integrates with Active Directory, so it doesn't work if you
don't want to join your workstation to the Kerberos realm. Or as in
Stephens case, you want to be *both* on the Active Directory and in a
non-trusted Unix Kerberos realm.


But we're talking two different issues. Deprecating/removing krb5 is a
different thing from having GSSAPI and SSPI mutually exclusive or not.

//Magnus

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] Future of krb5 authentication

[Gregory Stark]

"Tom Lane" <[EMAIL PROTECTED]> writes:

> The real problem in my mind is this business of the gssapi and krb5
> support being mutually exclusive.  

Oh, I didn't catch that. That's wrong anyways, there could be multiple
applications on the same machine, some of which use krb4 and some which use
gssapi.

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com


---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match

Re: [HACKERS] Future of krb5 authentication

[Joshua D. Drake]

Joshua D. Drake wrote:

Dave Page wrote:

Andrew Dunstan wrote:



Dave Page wrote:

Magnus Hagander wrote:

libpq would still work against older server versions, right?


Not once krb5 is removed. Assuming the older server version used 
krb5 auth.


OK, well thats a problem. pgAdmin supports back to 7.3...




I think you need to put forward an alternative plan, then. Asking us 
to wait until your oldest version is 8.4 before we rip it out of 
libpq doesn't sound too good.


pgAdmin was just one example. This prevents anyone with kerberos5 in a 
similar situation upgrading their client libraries - including users 
of the myriad of apps that use psqlODBC.


Who likely don't use kerberos.


I would also note that Magnus's proposal means that krb5 won't be 
removed for *at least* 2 years. I think that is plenty of time for an 
EOL cycle on a feature.


Joshua D. Drake




Joshua D. Drake



Regards, Dave

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

   http://www.postgresql.org/about/donate




---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

  http://www.postgresql.org/docs/faq




---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

   http://www.postgresql.org/about/donate

Re: [HACKERS] Future of krb5 authentication

[Tom Lane]

Magnus Hagander <[EMAIL PROTECTED]> writes:
> On Wed, Jul 18, 2007 at 10:46:58AM -0400, Tom Lane wrote:
>> This needs to be fixed.

> Non, GSSAPI and krb5 are *not* mutually exclusive.

> SSPI and GSSAPI are mutually exclusive.

Color me confused then.  What's the difference?

regards, tom lane

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Re: [HACKERS] Comments on the HOT design

[Simon Riggs]

On Wed, 2007-07-18 at 11:41 +0100, Heikki Linnakangas wrote:

> I've done some experimenting on those items, producing several badly
> broken versions of the patch partly implementing those ideas. It looks
> like the patch size will go down from ~240 kB to ~210 kB, and more
> importantly, there will be less new concepts and states a tuple can be
> in and less WAL record types.

I'm not certain Pavan is right, nor am I certain Heikki is, we may find
that both techniques are needed at various times and so we might need
both code paths. 

So I think we need ways of measuring the dynamic behaviour of the patch.
A trace_hot parameter, or similar, is needed to produce additional
information. Or something...

We can then go for the simplest version of the patch into beta, with
beta testers posting their trace info back to allow us to evaluate the
more complex options. The quicker we get to beta the better.

-- 
  Simon Riggs
  EnterpriseDB  http://www.enterprisedb.com


---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Re: [HACKERS] What is the maximum encoding-conversion growth rate, anyway?

[Bruce Momjian]

This has been saved for the 8.4 release:

http://momjian.postgresql.org/cgi-bin/pgpatches_hold

---

Tatsuo Ishii wrote:
> The conclusion of the discussion appears that we could reduce
> MAX_CONVERSION_GROWTH from 4 to 3 safely with all existing built-in
> conversions.
> 
> However, since user defined conversions could set arbitrary growth
> rate, probably it would be better leave it as it is now.
> 
> For 8.4, maybe we could change conversion function's signature so that
> we don't need to have the fixed conversion rate as Tom suggested.
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> 
> > Where are we on this?
> > 
> > ---
> > 
> > Tom Lane wrote:
> > > I just rearranged the code in mbutils.c a little bit to make it more
> > > robust if conversion of an over-length string is attempted, and noted
> > > this comment:
> > > 
> > > /*
> > >  * When converting strings between different encodings, we assume that 
> > > space
> > >  * for converted result is 4-to-1 growth in the worst case. The rate for
> > >  * currently supported encoding pairs are within 3 (SJIS JIS X0201 half 
> > > width
> > >  * kanna -> UTF8 is the worst case).  So "4" should be enough for the 
> > > moment.
> > >  *
> > >  * Note that this is not the same as the maximum character width in any
> > >  * particular encoding.
> > >  */
> > > #define MAX_CONVERSION_GROWTH  4
> > > 
> > > It strikes me that this is overly pessimistic, since we do not support
> > > 5- or 6-byte UTF8 characters, and AFAICS there are no 1-byte characters
> > > in any supported encoding that require 4 bytes in another.  Could we
> > > reduce the multiplier to 3?  Or even 2?  This has a direct impact on the
> > > longest COPY lines we can support, so I'd like it not to be larger than
> > > necessary.
> > > 
> > >   regards, tom lane
> > > 
> > > ---(end of broadcast)---
> > > TIP 4: Have you searched our list archives?
> > > 
> > >http://archives.postgresql.org
> > 
> > -- 
> >   Bruce Momjian  <[EMAIL PROTECTED]>  http://momjian.us
> >   EnterpriseDB   http://www.enterprisedb.com
> > 
> >   + If your life is a hard drive, Christ can be your backup. +

-- 
  Bruce Momjian  <[EMAIL PROTECTED]>  http://momjian.us
  EnterpriseDB   http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

On Wed, Jul 18, 2007 at 10:46:58AM -0400, Tom Lane wrote:
> Magnus Hagander <[EMAIL PROTECTED]> writes:
> > But sure, we might leave it in there until there's a direct problem with it
> > (other than the ones we already know). Can I still get my deprecation of it
> > though? ;-)
> 
> In the krb4 case, we left it in there until there was very little
> probability anyone was using it anymore, and AFAIR there was no
> significant maintenance burden from that.  I don't see a reason to be
> harsher on the krb5 case.
> 
> The real problem in my mind is this business of the gssapi and krb5
> support being mutually exclusive.  That is going to cause tremendous
> pain because there won't be any convenient upgrade path.  Particularly
> not for users of binary packages (RPMs etc) --- they'll be screwed
> if their packager changes, and have no way to upgrade if he doesn't.
> This needs to be fixed.

Non, GSSAPI and krb5 are *not* mutually exclusive.

SSPI and GSSAPI are mutually exclusive.

You can use krb5 and GSSAPI or krb5 and SSPI just fine.

//Magnus

---(end of broadcast)---
TIP 6: explain analyze is your friend

Re: [HACKERS] Future of krb5 authentication

[Chris Browne]

[EMAIL PROTECTED] (Peter Eisentraut) writes:
> Am Mittwoch, 18. Juli 2007 13:21 schrieb Magnus Hagander:
>> The main reasons would be to have less code to maintain,
>
> I don't think the krb5 support has needed all that much maintenance in the 
> last few years.
>
>> and to make life 
>> easier for packagers. For example, win32 would no longer need to ship the
>> kerberos binaries in the package (and update it properly etc).
>
> Nobody is forcing anyone to ship anything anyway.

But on the other hand, if support for something gets "ripped out,"
that pretty much forces packagers to NOT ship it anymore.  So it's not
totally symmetrical...

>> But sure, we might leave it in there until there's a direct problem with it
>> (other than the ones we already know). Can I still get my deprecation of it
>> though? ;-)
>
> You might want to inquire about that on -general.

Good idea...

-- 
let name="cbbrowne" and tld="linuxfinances.info" in String.concat "@" 
[name;tld];;
http://www3.sympatico.ca/cbbrowne/internet.html
"This must be Thursday.  I never could get the hang of Thursdays."
- Arthur Dent

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] Future of krb5 authentication

[Tom Lane]

Magnus Hagander <[EMAIL PROTECTED]> writes:
> But sure, we might leave it in there until there's a direct problem with it
> (other than the ones we already know). Can I still get my deprecation of it
> though? ;-)

In the krb4 case, we left it in there until there was very little
probability anyone was using it anymore, and AFAIR there was no
significant maintenance burden from that.  I don't see a reason to be
harsher on the krb5 case.

The real problem in my mind is this business of the gssapi and krb5
support being mutually exclusive.  That is going to cause tremendous
pain because there won't be any convenient upgrade path.  Particularly
not for users of binary packages (RPMs etc) --- they'll be screwed
if their packager changes, and have no way to upgrade if he doesn't.
This needs to be fixed.

regards, tom lane

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Re: [HACKERS] Future of krb5 authentication

[Dave Page]

Joshua D. Drake wrote:
pgAdmin was just one example. This prevents anyone with kerberos5 in a 
similar situation upgrading their client libraries - including users 
of the myriad of apps that use psqlODBC.


Who likely don't use kerberos.


Probably not in the majority of cases - but we have a large userbase 
these days, and a small percentage may still equate to a large number. I 
know at least two people that do use psqlODBC + Kerberos.


Regards, Dave

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match

Re: [HACKERS] Future of krb5 authentication

[Joshua D. Drake]

Dave Page wrote:

Andrew Dunstan wrote:



Dave Page wrote:

Magnus Hagander wrote:

libpq would still work against older server versions, right?


Not once krb5 is removed. Assuming the older server version used 
krb5 auth.


OK, well thats a problem. pgAdmin supports back to 7.3...




I think you need to put forward an alternative plan, then. Asking us 
to wait until your oldest version is 8.4 before we rip it out of libpq 
doesn't sound too good.


pgAdmin was just one example. This prevents anyone with kerberos5 in a 
similar situation upgrading their client libraries - including users of 
the myriad of apps that use psqlODBC.


Who likely don't use kerberos.

Joshua D. Drake



Regards, Dave

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

   http://www.postgresql.org/about/donate




---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

  http://www.postgresql.org/docs/faq

Re: [HACKERS] Future of krb5 authentication

[Joshua D. Drake]

Dave Page wrote:

Magnus Hagander wrote:

libpq would still work against older server versions, right?


Not once krb5 is removed. Assuming the older server version used krb5 
auth.


OK, well thats a problem. pgAdmin supports back to 7.3...


How many people actually use kerberos... How many people who are using 
kerberos are going to be running 7.3. 7.3 is no longer supported so by 
postgresql.org so who cares.


Joshua D. Drake



/D

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings




---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] Future of krb5 authentication

[Peter Eisentraut]

Am Mittwoch, 18. Juli 2007 13:21 schrieb Magnus Hagander:
> The main reasons would be to have less code to maintain,

I don't think the krb5 support has needed all that much maintenance in the 
last few years.

> and to make life 
> easier for packagers. For example, win32 would no longer need to ship the
> kerberos binaries in the package (and update it properly etc).

Nobody is forcing anyone to ship anything anyway.

> But sure, we might leave it in there until there's a direct problem with it
> (other than the ones we already know). Can I still get my deprecation of it
> though? ;-)

You might want to inquire about that on -general.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq

Re: [HACKERS] Updated tsearch documentation

[Oleg Bartunov]

On Tue, 17 Jul 2007, Bruce Momjian wrote:


Oleg Bartunov wrote:

On Tue, 17 Jul 2007, Bruce Momjian wrote:


I think the tsearch documentation is nearing completion:

http://momjian.us/expire/fulltext/HTML/textsearch.html

but I am not happy with how tsearch is enabled in a user table:

http://momjian.us/expire/fulltext/HTML/textsearch-app-tutorial.html

Aside from the fact that it needs more examples, it only illustrates an
example where someone creates a table, populates it, then adds a
tsvector column, populates that, then creates an index.

That seems quite inflexible.  Is there a way to avoid having a separate
tsvector column?  What happens if the table is dynamic?  How is that
column updated based on table changes?  Triggers?  Where are the
examples?  Can you create an index like this:


I agree, that there are could be more examples, but text search doesn't
require something special !
*Example* of trigger function is documented on
http://momjian.us/expire/fulltext/HTML/textsearch-opfunc.html


Yes, I see that in tsearch() here:

http://momjian.us/expire/fulltext/HTML/textsearch-opfunc.html#TEXTSEARC$

I assume my_filter_name is optional right?  I have updated the prototype
to be:

tsearch([vector_column_name], [my_filter_name], text_column_name [, ... 
])

Is this accurate?  What does this text below it mean?


no, this in inaccurate. First, vector_column_name is not optional argument,
it's a name of tsvector column name.



There can be many functions and text columns specified in a tsearch()
trigger. The following rule is used: a function is applied to all
subsequent TEXT columns until the next matching column occurs.


The idea, is to provide user to preprocess text before applying 
tsearch machinery. my_filter_name() preprocess text_column_name1,

text_column_name2,
The original syntax allows to specify for every text columns their 
preprocessing functions.


So, I suggest to keep original syntax, change 'vector_column_name' to
'tsvector_column_name'.



Why are we allowing my_filter_name here?  Isn't that something for a
custom trigger.  Is calling it tsearch() a good idea?  Why not
tsvector_trigger().


I don't see any benefit from the tsvector_trigger() name. If you want to add
some semantic, than tsvector_update_trigger() would be better.  Anyway,
this trigger is an illustration.




CREATE INDEX textsearch_id ON pgweb USING gin(to_tsvector(column));

That avoids having to have a separate column because you can just say:

WHERE to_query('XXX') @@ to_tsvector(column)


yes, it's possible, but without ranking, since currently it's impossible
to store any information in index (it's pg's feature). btw, this should
works and for GiST index also.


What if they use @@@.  Wouldn't that work because it is going to check
the heap?


It would work, it'd recalculate to_tsvector(column) for rows found
( for GiST - to remove false hits and for weight information, for 
GIN - for weight information only).





That kind of search is useful if there is  another natural ordering of search
results, for example, by timestamp.



How do we make sure that the to_query is using the same text search
configuration as the 'column' or index?  Perhaps we should suggest:


please, keep in mind, it's not mandatory to use the same configuration
at search time, that was used at index creation.


Well, sort of.  If you have stop words in the tquery configuration, you
aren't going to hit any matches in the tsvector, right?  Same for
synonymns, I suppose.  I can see that stemming would work if there was a
mismatch between tsquery and tsvector.


 CREATE INDEX textsearch_idx ON pgweb USING gin(to_tsvector('english',column));

so that at least the configuration is documented in the index.


yes, it's better to always explicitly specify configuration name and not
rely on default configuration.
Unfortunately, configuration name doesn't saved in the index.


as Teodor corrected me, index doesn't know about configuration at all !
What accurate user could do, is to provide configuration name in the 
comment for tsvector column. Configuration name is an accessory of

to_tsvector() function.

In principle, tsvector as any data type could be obtained by any other ways,
for example, OpenFTS construct tsvector following its own rules.



I was more concerned that there is nothing documenting the configuration
used by the index or the tsvector table column trigger.  By doing:


again, index has nothing with configuration name.
Our trigger function is an example, which uses default configuration name.
User could easily write it's own trigger to keep tsvector column up to date 
and use configuration name as a parameter.




CREATE INDEX textsearch_idx ON pgweb USING 
gin(to_tsvector('english',column));

you guarantee that the index uses 'english' for all its entries.  If you
omit the 'english' or use a different configuration, it will heap scan
the table,

[HACKERS] Interaction between user-defined array types and cluster.c:rebuild_relation()

[Gregory Stark]

Just an observation, but when we cluster (or any other time we rebuild a heap)
we are creating a useless record type and array type for the temporary heap we
use which is subsequently dropped. 

postgres=# cluster;
DEBUG:  drop auto-cascades to type pg_temp_16474
DEBUG:  drop auto-cascades to type pg_temp_16474[]
DEBUG:  drop auto-cascades to type pg_temp_16471
DEBUG:  drop auto-cascades to type pg_temp_16471[]

I can't think of any serious negative effects this could have so perhaps it's
not worth worrying about. The worst I can think of is the catalog bloat but
we're already bloating pg_class and pg_attribute anyways so this doesn't seem
like a big deal.

But I thought I would mention it in case anyone else is offended or knows of a
more serious problem it could cause.

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com


---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly

Re: [HACKERS] Future of krb5 authentication

[Dave Page]

Andrew Dunstan wrote:



Dave Page wrote:

Magnus Hagander wrote:

libpq would still work against older server versions, right?


Not once krb5 is removed. Assuming the older server version used krb5 
auth.


OK, well thats a problem. pgAdmin supports back to 7.3...




I think you need to put forward an alternative plan, then. Asking us to 
wait until your oldest version is 8.4 before we rip it out of libpq 
doesn't sound too good.


pgAdmin was just one example. This prevents anyone with kerberos5 in a 
similar situation upgrading their client libraries - including users of 
the myriad of apps that use psqlODBC.


Regards, Dave

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

   http://www.postgresql.org/about/donate

Re: [HACKERS] Future of krb5 authentication

[Andrew Dunstan]

Dave Page wrote:

Magnus Hagander wrote:

libpq would still work against older server versions, right?


Not once krb5 is removed. Assuming the older server version used krb5 
auth.


OK, well thats a problem. pgAdmin supports back to 7.3...




I think you need to put forward an alternative plan, then. Asking us to 
wait until your oldest version is 8.4 before we rip it out of libpq 
doesn't sound too good.


cheers

andrew

---(end of broadcast)---
TIP 6: explain analyze is your friend

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

On Wed, Jul 18, 2007 at 12:26:28PM +0100, Heikki Linnakangas wrote:
> Magnus Hagander wrote:
> > But sure, we might leave it in there until there's a direct problem with it
> > (other than the ones we already know). Can I still get my deprecation of it
> > though? ;-)
> 
> I'm not sure what the deprecation would mean in the client-side. You're
> going to need it if you want to connect to a server that uses it,
> there's no alternative.

No, it would mean nothing on the client.

> In the server, I don't see a problem with it. What would the deprecation
> mean, though? Mention in the docs that it's going to removed sometime in
> the future? A warning if you enable it?

At least a warning in the docs. And possibly also a warning when you enable
it, depending on what people thing is appropriate.

//Magnus

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly

Re: [HACKERS] Future of krb5 authentication

[Heikki Linnakangas]

Magnus Hagander wrote:
> But sure, we might leave it in there until there's a direct problem with it
> (other than the ones we already know). Can I still get my deprecation of it
> though? ;-)

I'm not sure what the deprecation would mean in the client-side. You're
going to need it if you want to connect to a server that uses it,
there's no alternative.

In the server, I don't see a problem with it. What would the deprecation
mean, though? Mention in the docs that it's going to removed sometime in
the future? A warning if you enable it?

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

On Wed, Jul 18, 2007 at 12:16:49PM +0100, Heikki Linnakangas wrote:
> Magnus Hagander wrote:
> > On Wed, Jul 18, 2007 at 11:57:19AM +0100, Dave Page wrote:
> >> Magnus Hagander wrote:
>  libpq would still work against older server versions, right?
> >>> Not once krb5 is removed. Assuming the older server version used krb5 
> >>> auth.
> >> OK, well thats a problem. pgAdmin supports back to 7.3...
> > 
> > You have a similar problem there already - 8.1 removed support for Kerberos
> > 4, so if your 7.3 server is configged with krb4, you loose anyway.
> 
> Let's be practical here. We're going to have support for both in libpq
> for at least one version anyway. What do we gain by removing that
> support in a later release? I think we should just keep it around until
> we have a pressing reason to remove it, say if it gets in the way of
> implementing some new feature.
> 
> In the server, I think we could remove it sooner. But even there, is
> there a reason why we should?

The main reasons would be to have less code to maintain, and to make life
easier for packagers. For example, win32 would no longer need to ship the
kerberos binaries in the package (and update it properly etc).

But sure, we might leave it in there until there's a direct problem with it
(other than the ones we already know). Can I still get my deprecation of it
though? ;-)

//Magnus


---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Re: [HACKERS] Future of krb5 authentication

[Heikki Linnakangas]

Magnus Hagander wrote:
> On Wed, Jul 18, 2007 at 11:57:19AM +0100, Dave Page wrote:
>> Magnus Hagander wrote:
 libpq would still work against older server versions, right?
>>> Not once krb5 is removed. Assuming the older server version used krb5 auth.
>> OK, well thats a problem. pgAdmin supports back to 7.3...
> 
> You have a similar problem there already - 8.1 removed support for Kerberos
> 4, so if your 7.3 server is configged with krb4, you loose anyway.

Let's be practical here. We're going to have support for both in libpq
for at least one version anyway. What do we gain by removing that
support in a later release? I think we should just keep it around until
we have a pressing reason to remove it, say if it gets in the way of
implementing some new feature.

In the server, I think we could remove it sooner. But even there, is
there a reason why we should?

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] Future of krb5 authentication

[Dave Page]

Magnus Hagander wrote:

On Wed, Jul 18, 2007 at 11:57:19AM +0100, Dave Page wrote:

Magnus Hagander wrote:

libpq would still work against older server versions, right?

Not once krb5 is removed. Assuming the older server version used krb5 auth.

OK, well thats a problem. pgAdmin supports back to 7.3...


You have a similar problem there already - 8.1 removed support for Kerberos
4, so if your 7.3 server is configged with krb4, you loose anyway.


We never shipped krb4 support in any of our self contained binary 
distros (I'm not counting those which rely on external libpq packages).


Regards, Dave

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

On Wed, Jul 18, 2007 at 11:57:19AM +0100, Dave Page wrote:
> Magnus Hagander wrote:
> >>libpq would still work against older server versions, right?
> >
> >Not once krb5 is removed. Assuming the older server version used krb5 auth.
> 
> OK, well thats a problem. pgAdmin supports back to 7.3...

You have a similar problem there already - 8.1 removed support for Kerberos
4, so if your 7.3 server is configged with krb4, you loose anyway.

//Magnus

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] Future of krb5 authentication

[Dave Page]

Magnus Hagander wrote:

libpq would still work against older server versions, right?


Not once krb5 is removed. Assuming the older server version used krb5 auth.


OK, well thats a problem. pgAdmin supports back to 7.3...

/D

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] Future of krb5 authentication

[Magnus Hagander]

On Wed, Jul 18, 2007 at 11:45:19AM +0100, Heikki Linnakangas wrote:
> Magnus Hagander wrote:
> > Now that we have working GSSAPI authentication, I'd like to see the
> > following done:
> > 
> > * Deprecate krb5 authentication in 8.3. At least in documentation, possibly
> > with a warning when loading pg_hba.conf?
> > * Remove krb5 authenticatino completely in 8.4.
> 
> libpq would still work against older server versions, right?

Not once krb5 is removed. Assuming the older server version used krb5 auth.
I want to remove it from both libpq and the server.
(8.3 libpq would of course work with older versions, since it's only
deprecated at that point)

I guess a compromise would be to remove it from the server in 8.4 and libpq
in 8.5 or so, if people think that it's a problem. But I think we definitly
want to get it out of libpq as well eventually.

//Magnus

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Re: [HACKERS] Future of krb5 authentication

[Heikki Linnakangas]

Magnus Hagander wrote:
> Now that we have working GSSAPI authentication, I'd like to see the
> following done:
> 
> * Deprecate krb5 authentication in 8.3. At least in documentation, possibly
> with a warning when loading pg_hba.conf?
> * Remove krb5 authenticatino completely in 8.4.

libpq would still work against older server versions, right?

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org

[HACKERS] Comments on the HOT design

[Heikki Linnakangas]

Here's what I think we should do to the HOT patch:

1. Get rid of row-level fragmentation and handling dealing with
LP_DELETEd line pointers. Instead, take a vacuum lock opportunistically,
and defrag pages using the normal PageRepairFragmentation function. I'm
not sure where exactly we would do the pruning and where we would call
PageRepairFragmentation. We could do it in heap_release_fetch, but we
need some logic to decide when it's helpful and when it's a waste of time.

2. Get rid of separate handling and WAL record types for pruning aborted
tuples, ISTM that's no longer needed with the simplified pruning method.

3. Currently, the patch has a separate code path for pruning pages in
VACUUM FULL, which removes any DEAD tuples in the middle of chains, and
fixes the ctid/xmin of the previous/next tuple to keep the chain valid.
Instead, we should just prune all the preceding tuples in the chain,
since they're in fact dead as well. Our simplistic check with OldestXmin
just isn't enough to notice that. I think we can share most of the code
between normal pruning and VACUUM FULL, which is good because the VACUUM
FULL codepath is used very seldom, so if there's any bugs in there they
might go unnoticed for a long time.

4. Write only one WAL record per pruned page, instead of one per update
chain.

I've done some experimenting on those items, producing several badly
broken versions of the patch partly implementing those ideas. It looks
like the patch size will go down from ~240 kB to ~210 kB, and more
importantly, there will be less new concepts and states a tuple can be
in and less WAL record types.

I know we've been running DBT-2 tests with the patch, and that it's
effective in reducing the need to vacuum the big tables which gives
better throughput in long runs. But I also know that a lot of people are
interested in the potential to avoid CPU overhead of index inserts. We
need to run CPU bound benchmarks to measure that effect as well.

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq

Re: [HACKERS] SSPI authentication

[Magnus Hagander]

On Tue, Jul 17, 2007 at 11:00:35AM -0700, Paul Silveira wrote:
> 
> This is great.  I've worked on 2 projects in the last year that desperately
> needed this.  It will certainly make the security model more seamless...

Thanks for letting us know.

Are you interested in just the SSPI parts, or also in being able to use
both SSPI and GSSAPI at the same time?

//Magnus

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org

[HACKERS] Future of krb5 authentication

[Magnus Hagander]

Now that we have working GSSAPI authentication, I'd like to see the
following done:

* Deprecate krb5 authentication in 8.3. At least in documentation, possibly
with a warning when loading pg_hba.conf?
* Remove krb5 authenticatino completely in 8.4.

The reasons for this is:
* krb5 auth doesn't do anything that gssapi doesn't.
* krb5 authentication doesn't follow a published standard. It follows API
examples from MIT later copied by Heimdal, but there is no documented
standard.
* krb5 authentication operates directly on the socket and as such violates
the libpq protocol. This means it's not protected by SSL if you have SSL on
your connection, and that it may misbehave with async sockets.


This was actually on the agenda when we first talked about doig gssapi, but
now that we have it it's time to bring it up again...

Comments?

//Magnus


---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly

Re: [HACKERS] SSPI authentication

[Magnus Hagander]

On Wed, Jul 18, 2007 at 09:44:02AM +0100, Dave Page wrote:
> Magnus Hagander wrote:
> > Dave Page wrote:
> >> Magnus Hagander wrote:
> >>> So what we'd need in that case is a new libpq connectionstring
> >>> parameter. Which can be done, but it'd require that all frontends that
> >>> use libpq add support for it - such as pgadmin. I'm not sure if the ODBC
> >>> driver will support arbitrary arguments, otherwise that one needs it too.
> >>>
> >>> As I'm sure you can tell, I'm far from convinced this is a good idea ;-)
> >>> Anybody else want to comment on this?
> >> The ODBC driver would need modification (as would pgAdmin of course).
> >> Whats more of a concern is that we already have ODBC connection strings
> >> that can be too long - adding yet another option will make that worse of
> >> course.
> > 
> > Interesting, didn't know that. That makes that option even less interesting.
> > 
> > Can you comment on if the current ODBC driver will pick up GSSAPI
> > authentication from libpq or if it needs new code to deal with it? I
> > never quite figured out how they integrate with libpq for the
> > authentication part since it moved away from using libpq for everything
> > again.
> 
> It should 'just work' I guess - it does for the existing Kerberos
> support. I never really studied GSSAPI though so I may be missing some
> fundamental point.

Ok, I got around to do some testing and it works perfectly fine. At least
on Windows - don't have ODBC set up anywhere else. But on windows + the
SSPI patch I just posted I get integrated login without any problems from a
vbscript using ODBC. Driver version 8.1 even :-)

//Magnus

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org

Re: [HACKERS] What is the maximum encoding-conversion growth rate, anyway?

[Tatsuo Ishii]

The conclusion of the discussion appears that we could reduce
MAX_CONVERSION_GROWTH from 4 to 3 safely with all existing built-in
conversions.

However, since user defined conversions could set arbitrary growth
rate, probably it would be better leave it as it is now.

For 8.4, maybe we could change conversion function's signature so that
we don't need to have the fixed conversion rate as Tom suggested.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

> Where are we on this?
> 
> ---
> 
> Tom Lane wrote:
> > I just rearranged the code in mbutils.c a little bit to make it more
> > robust if conversion of an over-length string is attempted, and noted
> > this comment:
> > 
> > /*
> >  * When converting strings between different encodings, we assume that space
> >  * for converted result is 4-to-1 growth in the worst case. The rate for
> >  * currently supported encoding pairs are within 3 (SJIS JIS X0201 half 
> > width
> >  * kanna -> UTF8 is the worst case).  So "4" should be enough for the 
> > moment.
> >  *
> >  * Note that this is not the same as the maximum character width in any
> >  * particular encoding.
> >  */
> > #define MAX_CONVERSION_GROWTH  4
> > 
> > It strikes me that this is overly pessimistic, since we do not support
> > 5- or 6-byte UTF8 characters, and AFAICS there are no 1-byte characters
> > in any supported encoding that require 4 bytes in another.  Could we
> > reduce the multiplier to 3?  Or even 2?  This has a direct impact on the
> > longest COPY lines we can support, so I'd like it not to be larger than
> > necessary.
> > 
> > regards, tom lane
> > 
> > ---(end of broadcast)---
> > TIP 4: Have you searched our list archives?
> > 
> >http://archives.postgresql.org
> 
> -- 
>   Bruce Momjian  <[EMAIL PROTECTED]>  http://momjian.us
>   EnterpriseDB   http://www.enterprisedb.com
> 
>   + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] What is the maximum encoding-conversion growth rate, anyway?

[Tatsuo Ishii]

Sorry for dealy.

> On Tue, May 29, 2007 20:51, Tatsuo Ishii wrote:
> 
> > Thinking more, it striked me that users can define arbitarily growing
> > rate by using CFREATE CONVERSION. So it seems we need functionality to
> > define the growing rate anyway.
> 
> Would it make sense to define just the longest and shortest character
> lengths for an encoding?  Then for any conversion you'd have a safe
> estimate of
> 
>   ceil(target_encoding.max_char_len / source_encoding.min_char_len)
> 
> ...without going through every possible conversion.

This will not work since certain CONVERSION allows n char to m char
conversion.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] SetBufferCommitInfoNeedsSave and race conditions

[Simon Riggs]

On Wed, 2007-07-18 at 01:01 +0100, Gregory Stark wrote:
> "Bruce Momjian" <[EMAIL PROTECTED]> writes:
> 
> > Where are we on this?
> 
> Well Simon just sent the reworked patch yesterday so the answer is we haven't
> started tuning this parameter. (Bruce's message is referring to the discussion
> about what the optimal value of lsns per clog page would be.)

>From discussion, Greg wishes to see a certain parameter higher, though
I'm fairly comfortable with the value now. Thanks to Greg for making me
think this through in more detail. There is some confirmation required,
but no significant tuning effort required.

Here's the thinking:

When we commit, we store an LSN that covers a group of transactions.
When we set hint bits we check the appropriate LSN. If it has been
flushed, then we set the hint bit.

If we're using synch commits then we can always set hint bits.
If we're mixing async commits with sync commits then the clog LSNs will
very often be flushed already, so we'll be able to set most hint bits.

If we are doing *only* simple async commits then only the WAL writer
flushes WAL. In that case when we try to set hint bits we will only be
prevented from setting hint bits for the last "few" transactions. So how
many is that exactly?

We are flushing WAL every W seconds and doing T transactions per second,
then we will flush WAL every T*W transactions. e.g. W = 0.2 (default), T
~ 10,000 => T*W = 2,000 transactions.

We have divided up each clog page up into N groups of transactions, each
with their own LSN, then we will have 32,000/N transactions per group.

The best situation is that we set N such that we never defer the setting
of a hint bit for a transaction that has been flushed.
There is no point setting N any lower than
32,000/N = T*W
i.e. N <= W * (32,000/T)

My laptop can do T=1000 with W=0.2 with pgbench, so N <= 6. 

By definition, anyone using async commit will have a high transaction
rate (else there is no benefit), so N seems able to be reasonably small.
If the transaction rate drops off momentarily the number of unflushed
committed async transactions doesn't rise, it falls quickly to zero as
the WAL writer flushes WAL up to the last async LSN.

The patch sets a value of 16, which seems likely to be enough to cover
most situations. 

Having thought that through, I see two changes I'd like to make:

- I realise that I've arranged the LSNs to be striped rather than
grouped. I'll change this part of the patch - couple of lines in
GetLSNIndex().

- The LSN of sync commits is also used to update the LSN of the clog
page. That is unnecessary and we only need set the clog LSN for async
commits.

I'll wait for other review comments before cutting version 23

-- 
  Simon Riggs
  EnterpriseDB  http://www.enterprisedb.com


---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly

Re: [HACKERS] SSPI authentication

[Dave Page]

Magnus Hagander wrote:
> Dave Page wrote:
>> Magnus Hagander wrote:
>>> So what we'd need in that case is a new libpq connectionstring
>>> parameter. Which can be done, but it'd require that all frontends that
>>> use libpq add support for it - such as pgadmin. I'm not sure if the ODBC
>>> driver will support arbitrary arguments, otherwise that one needs it too.
>>>
>>> As I'm sure you can tell, I'm far from convinced this is a good idea ;-)
>>> Anybody else want to comment on this?
>> The ODBC driver would need modification (as would pgAdmin of course).
>> Whats more of a concern is that we already have ODBC connection strings
>> that can be too long - adding yet another option will make that worse of
>> course.
> 
> Interesting, didn't know that. That makes that option even less interesting.
> 
> Can you comment on if the current ODBC driver will pick up GSSAPI
> authentication from libpq or if it needs new code to deal with it? I
> never quite figured out how they integrate with libpq for the
> authentication part since it moved away from using libpq for everything
> again.

It should 'just work' I guess - it does for the existing Kerberos
support. I never really studied GSSAPI though so I may be missing some
fundamental point.

/D


---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings