Re: [HACKERS] unaccent module - two params function should be immutable

2013-11-18 Thread Bruce Momjian
On Fri, Nov 8, 2013 at 06:00:53PM -0500, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: [ mark unaccent functions immutable ] Applied. This patch is flat out wrong and needs to be reverted. The functions were correctly marked (by you!) in commit

Re: [HACKERS] unaccent module - two params function should be immutable

2013-11-08 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: [ mark unaccent functions immutable ] Applied. This patch is flat out wrong and needs to be reverted. The functions were correctly marked (by you!) in commit c0577c92a84cc477a88fe6868c16c4a7e3348b11 on the basis of the discussion of bug #5781,

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Bruce Momjian
On Tue, Sep 24, 2013 at 05:36:58PM -0400, Bruce Momjian wrote: On Tue, Sep 17, 2013 at 10:15:47AM -0400, Robert Haas wrote: On Sat, Sep 14, 2013 at 9:42 AM, Pavel Stehule pavel.steh...@gmail.com wrote: I have developed the attached patch based on your suggestion. I did not see

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Pavel Stehule
2013/10/8 Bruce Momjian br...@momjian.us On Tue, Sep 24, 2013 at 05:36:58PM -0400, Bruce Momjian wrote: On Tue, Sep 17, 2013 at 10:15:47AM -0400, Robert Haas wrote: On Sat, Sep 14, 2013 at 9:42 AM, Pavel Stehule pavel.steh...@gmail.com wrote: I have developed the attached patch based

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Bruce Momjian
On Tue, Oct 8, 2013 at 06:31:03PM +0200, Pavel Stehule wrote: 2013/10/8 Bruce Momjian br...@momjian.us On Tue, Sep 24, 2013 at 05:36:58PM -0400, Bruce Momjian wrote: On Tue, Sep 17, 2013 at 10:15:47AM -0400, Robert Haas wrote: On Sat, Sep 14, 2013 at 9:42 AM, Pavel

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Pavel Stehule
2013/10/8 Bruce Momjian br...@momjian.us On Tue, Oct 8, 2013 at 06:31:03PM +0200, Pavel Stehule wrote: 2013/10/8 Bruce Momjian br...@momjian.us On Tue, Sep 24, 2013 at 05:36:58PM -0400, Bruce Momjian wrote: On Tue, Sep 17, 2013 at 10:15:47AM -0400, Robert Haas wrote:

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Bruce Momjian
On Tue, Oct 8, 2013 at 06:38:30PM +0200, Pavel Stehule wrote: I am not sure - does pg_upgrade change of flag after upgrade without increasing version number? What happens in pg_upgrade is that the CREATE EXTENSION command is pg_dump'ed, and run by pg_uprade, and it then pulls from the SQL

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Pavel Stehule
2013/10/8 Bruce Momjian br...@momjian.us On Tue, Oct 8, 2013 at 06:38:30PM +0200, Pavel Stehule wrote: I am not sure - does pg_upgrade change of flag after upgrade without increasing version number? What happens in pg_upgrade is that the CREATE EXTENSION command is pg_dump'ed, and run

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Alvaro Herrera
Bruce Momjian escribió: Do we need to update any version or anything? I didn't think so. I think there should be an 1.1 version here. That way, if somebody is using the existing definition from the 1.0 module, they can get the new definition by doing an extension upgrade. -- Álvaro Herrera

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Bruce Momjian
On Tue, Oct 8, 2013 at 02:25:25PM -0300, Alvaro Herrera wrote: Bruce Momjian escribió: Do we need to update any version or anything? I didn't think so. I think there should be an 1.1 version here. That way, if somebody is using the existing definition from the 1.0 module, they can get

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Alvaro Herrera
Bruce Momjian escribió: On Tue, Oct 8, 2013 at 02:25:25PM -0300, Alvaro Herrera wrote: Bruce Momjian escribió: Do we need to update any version or anything? I didn't think so. I think there should be an 1.1 version here. That way, if somebody is using the existing definition

Re: [HACKERS] unaccent module - two params function should be immutable

2013-09-24 Thread Bruce Momjian
On Tue, Sep 17, 2013 at 10:15:47AM -0400, Robert Haas wrote: On Sat, Sep 14, 2013 at 9:42 AM, Pavel Stehule pavel.steh...@gmail.com wrote: I have developed the attached patch based on your suggestion. I did not see anything in the code that would make it STABLE, except a lookup of a

Re: [HACKERS] unaccent module - two params function should be immutable

2013-09-17 Thread Robert Haas
On Sat, Sep 14, 2013 at 9:42 AM, Pavel Stehule pavel.steh...@gmail.com wrote: I have developed the attached patch based on your suggestion. I did not see anything in the code that would make it STABLE, except a lookup of a dictionary library: dictOid =

Re: [HACKERS] unaccent module - two params function should be immutable

2013-09-14 Thread Pavel Stehule
2013/9/11 Bruce Momjian br...@momjian.us On Tue, Feb 19, 2013 at 08:30:29AM +0100, Pavel Stehule wrote: Hello There was a proposal to change flag of function to immutable - should be used in indexes CREATE FUNCTION unaccent(regdictionary, text) RETURNS text AS

Re: [HACKERS] unaccent module - two params function should be immutable

2013-09-10 Thread Bruce Momjian
On Tue, Feb 19, 2013 at 08:30:29AM +0100, Pavel Stehule wrote: Hello There was a proposal to change flag of function to immutable - should be used in indexes CREATE FUNCTION unaccent(regdictionary, text) RETURNS text AS 'MODULE_PATHNAME', 'unaccent_dict' LANGUAGE

Re: [HACKERS] Unaccent performance

2013-06-22 Thread Andres Freund
On 2013-06-21 22:52:04 +0100, Thom Brown wrote: CREATE OR REPLACE FUNCTION public.myunaccent(sometext text) RETURNS text LANGUAGE sql IMMUTABLE AS $function$ SELECT

[HACKERS] Unaccent performance

2013-06-21 Thread Thom Brown
Hi, The unaccent extension is great, especially with its customisability, but it's not always easy to recommend. I witnessed a customer using no less than 56 nested replace functions in an SQL function. I looked to see how much this can be mitigated by unaccent. It turns out that not all the

Re: [HACKERS] Unaccent performance

2013-06-21 Thread Thom Brown
On 21 June 2013 19:04, Thom Brown t...@linux.com wrote: Hi, The unaccent extension is great, especially with its customisability, but it's not always easy to recommend. I witnessed a customer using no less than 56 nested replace functions in an SQL function. I looked to see how much this

[HACKERS] unaccent module - two params function should be immutable

2013-02-18 Thread Pavel Stehule
Hello There was a proposal to change flag of function to immutable - should be used in indexes CREATE FUNCTION unaccent(regdictionary, text) RETURNS text AS 'MODULE_PATHNAME', 'unaccent_dict' LANGUAGE C STABLE STRICT; is there any progress? Regards Pavel Stehule --

Re: [HACKERS] unaccent extension missing some accents

2011-11-10 Thread Bruce Momjian
Tom Lane wrote: J Smith dark.panda+li...@gmail.com writes: I've attached a patch against master for unaccent.c that uses swscanf along with char2wchar and wchar2char instead of sscanf directly to initialize the unaccent extension and it appears to fix the problem in both the master and

Re: [HACKERS] unaccent extension missing some accents

2011-11-10 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Tom Lane wrote: However, the bigger picture is that OS X's UTF8 locales are broken through-and-through, and most of their other problems are not feasible to work around. If Apple's low-level code came from FreeBSD and NetBSD, how did they get so

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Tom Lane
J Smith dark.panda+li...@gmail.com writes: Alright, I wrote up another patch that uses strchr to parse out the lines of the unaccent.rules file, foregoing sscanf completely. Hopefully this looks a bit better than using swscanf. I looked at this a bit and realized that sscanf is actually doing

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Florian Pflug
On Nov7, 2011, at 17:46 , J Smith wrote: On Mon, Nov 7, 2011 at 11:12 AM, Tom Lane t...@sss.pgh.pa.us wrote: If OS X's UTF8 locales weren't so thoroughly broken (eg sorting does not work), I might be tempted to try to do it that way, but I still fail to see the point. After reviewing the code

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread J Smith
On Mon, Nov 7, 2011 at 11:12 AM, Tom Lane t...@sss.pgh.pa.us wrote: I looked at this a bit and realized that sscanf is actually doing a couple of critical things for us, which are lost in translation when doing it like this: 1. It ignores whitespace other than the dividing tab.  If we don't

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Tom Lane
J Smith dark.panda+li...@gmail.com writes: Anyways, lemme know if there's anything else I could help with or could test and whatnot. Cheers. If you have time to check that the patch I just committed fixes your problem, it'd be worth doing. I did not test it on OS X ...

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread J Smith
On Mon, Nov 7, 2011 at 11:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: If you have time to check that the patch I just committed fixes your problem, it'd be worth doing.  I did not test it on OS X ... Looks good to me, thanks. Would it even really be worth it to look into any of the other locale

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Tom Lane
J Smith dark.panda+li...@gmail.com writes: Would it even really be worth it to look into any of the other locale issues on OSX, given that PostgreSQL is now included in their default installs starting with 10.7, or would this really be more of a case of hoping Apple some day fixes the issue

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread J Smith
On Mon, Nov 7, 2011 at 11:53 AM, Florian Pflug f...@phlo.org wrote: Various issues with OSX and UTF-8 locales seems to come up quite often, yet we're not really in a position to do anything about them. Thus, I think we should warn about these issues and save people the trouble of finding out

[HACKERS] unaccent extension missing some accents

2011-11-06 Thread J Smith
G'day list. I've been messing around with the unaccent extension and I've noticed that some of the characters listed in the unaccent.rules file aren't actually being unaccented on my system. Here are the system details and whatnot. - OSX 10.7.2 - the server is compiled via macports. Tried

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread J Smith
Gah! Accidentally hit Send. Let me finish that last message before sending this time! G'day list. I've been messing around with the unaccent extension and I've noticed that some of the characters listed in the unaccent.rules file aren't actually being unaccented on my system. Here are the

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread Florian Pflug
On Nov6, 2011, at 18:43 , J Smith wrote: I put some elog debugging lines into unaccent.c and found that sscanf sometimes reads the scanned line by finding only one byte for the for the source character rather than the two required for the complete UTF-8 code point. It appears that the

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread J Smith
On Sun, Nov 6, 2011 at 1:18 PM, Florian Pflug f...@phlo.org wrote: What's the locale of the database you're seeing this in, and which charset does it use? I think scanf() uses isspace() and friends, and last time I looked the locale definitions where all pretty bogus on OSX. So maybe scanf()

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread Tom Lane
J Smith dark.panda+li...@gmail.com writes: I've attached a patch against master for unaccent.c that uses swscanf along with char2wchar and wchar2char instead of sscanf directly to initialize the unaccent extension and it appears to fix the problem in both the master and 9.1 branches. swscanf

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread J Smith
On 2011-11-06, at 7:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: swscanf doesn't seem like an acceptable approach: it's a function that is relied on nowhere else in PG, so it adds new portability risks of its own. It doesn't exist on some platforms that we support (like the one I'm typing this

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread J Smith
Alright, I wrote up another patch that uses strchr to parse out the lines of the unaccent.rules file, foregoing sscanf completely. Hopefully this looks a bit better than using swscanf. As for the other problems with isspace and such on OSX, it might be worth looking at the python portability

Re: [HACKERS] unaccent contrib

2011-09-26 Thread Oleg Bartunov
On Wed, 21 Sep 2011, Tom Lane wrote: Euler Taveira de Oliveira eu...@timbira.com writes: On 21-09-2011 13:28, Daniel VАzquez wrote: unaccent is compatible with postgresql 8.4 (but not is in their contrib version distribution) No, it is not. AFAICS it is necessary to add some backend code

Re: [HACKERS] unaccent contrib

2011-09-22 Thread Daniel Vázquez
Before 9.x, how do unaccent full text searches ? i 2011/9/21 Tom Lane t...@sss.pgh.pa.us Euler Taveira de Oliveira eu...@timbira.com writes: On 21-09-2011 13:28, Daniel Vázquez wrote: unaccent is compatible with postgresql 8.4 (but not is in their contrib version distribution) No, it

Re: [HACKERS] unaccent contrib

2011-09-22 Thread Euler Taveira de Oliveira
On 22-09-2011 12:39, Daniel Vázquez wrote: Before 9.x, how do unaccent full text searches ? Perform pre-processing (normalization) of the string *before* inserting and *before* searching. -- Euler Taveira de Oliveira - Timbira http://www.timbira.com.br/ PostgreSQL: Consultoria,

Re: [HACKERS] unaccent contrib

2011-09-22 Thread Robert Haas
2011/9/22 Daniel Vázquez daniel2d2...@gmail.com: Before 9.x, how do unaccent full text searches ? It seems that Oleg has published something on his web site that supposedly works with 8.4: http://www.sai.msu.su/~megera/wiki/unaccent But I'm not really sure how it works, or even where the

[HACKERS] unaccent contrib

2011-09-21 Thread Daniel Vázquez
Hi guys! I know Postgresql 9.x includes unaccent contrib on their deliver package. unaccent is compatible with postgresql 8.4 (but not is in their contrib version distribution) what's better way to setup unaccent module on Postgresql 8.4 production server. Copy contrib/unaccent from 9.x to the

Re: [HACKERS] unaccent contrib

2011-09-21 Thread Euler Taveira de Oliveira
On 21-09-2011 13:28, Daniel Vázquez wrote: unaccent is compatible with postgresql 8.4 (but not is in their contrib version distribution) No, it is not. AFAICS it is necessary to add some backend code that is not in 8.4. -- Euler Taveira de Oliveira - Timbira

Re: [HACKERS] unaccent contrib

2011-09-21 Thread Daniel Vázquez
... ok No alternatives for unaccent on 8.4? 2011/9/21 Euler Taveira de Oliveira eu...@timbira.com On 21-09-2011 13:28, Daniel Vázquez wrote: unaccent is compatible with postgresql 8.4 (but not is in their contrib version distribution) No, it is not. AFAICS it is necessary to add some

Re: [HACKERS] unaccent contrib

2011-09-21 Thread Euler Taveira de Oliveira
On 21-09-2011 15:23, Daniel Vázquez wrote: No alternatives for unaccent on 8.4? Not that I know of. -- Euler Taveira de Oliveira - Timbira http://www.timbira.com.br/ PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento -- Sent via pgsql-hackers mailing list

Re: [HACKERS] unaccent contrib

2011-09-21 Thread Devrim GÜNDÜZ
On Wed, 2011-09-21 at 18:28 +0200, Daniel Vázquez wrote: Can Global Development Group, make some acumulative rpm for contrib modules that are backward compatible??? No (as the RPM maintainer). -- Devrim GÜNDÜZ Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com PostgreSQL

Re: [HACKERS] unaccent contrib

2011-09-21 Thread Tom Lane
Euler Taveira de Oliveira eu...@timbira.com writes: On 21-09-2011 13:28, Daniel Vázquez wrote: unaccent is compatible with postgresql 8.4 (but not is in their contrib version distribution) No, it is not. AFAICS it is necessary to add some backend code that is not in 8.4. [ pokes at it ]

Re: [HACKERS] unaccent

2002-09-19 Thread Peter Eisentraut
[EMAIL PROTECTED] writes: Not to_ascii, since there are so many extended UNICODE characters that doesn't have any accent and should not be converted to an ASCII character. Really, the accent conversion should be part of the character set conversion routines. At least my local iconv does

[HACKERS] unaccent

2002-09-18 Thread nngodinh
Greetings, As far as I use the txtidx data structure in conjunction with gist indexing to make a word indexing of a very large UNICODE db, I've implemented a PostgreSQL function that uses libunac to unaccent TEXT fileds. The resulting text is in UTF-8, but you can modify it in the sources with

Re: [HACKERS] unaccent

2002-09-18 Thread Oleg Bartunov
On Wed, 18 Sep 2002 [EMAIL PROTECTED] wrote: Greetings, As far as I use the txtidx data structure in conjunction with gist indexing to make a word indexing of a very large UNICODE db, I've implemented a PostgreSQL function that uses libunac to unaccent TEXT fileds. The resulting text is

Re: [HACKERS] unaccent

2002-09-18 Thread nngodinh
functions as parameters. So my first idea was to integrate unac in tsearch. Bye. -- Messaggio Originale -- Date: Wed, 18 Sep 2002 15:08:59 +0300 (GMT) From: Oleg Bartunov [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: [HACKERS] unaccent On Wed, 18 Sep 2002 [EMAIL PROTECTED

Re: [HACKERS] unaccent

2002-09-18 Thread nngodinh
PROTECTED], [EMAIL PROTECTED] Subject: Re: [HACKERS] unaccent On Wed, Sep 18, 2002 at 03:08:59PM +0300, Oleg Bartunov wrote: On Wed, 18 Sep 2002 [EMAIL PROTECTED] wrote: Get libunac from: http://www.nongnu.org/unac/ (it uses iconv) Extract the archive, compile it (make). Move pg_unac.so

Re: [HACKERS] unaccent

2002-09-18 Thread Oleg Bartunov
On Wed, 18 Sep 2002, Karel Zak wrote: On Wed, Sep 18, 2002 at 03:08:59PM +0300, Oleg Bartunov wrote: On Wed, 18 Sep 2002 [EMAIL PROTECTED] wrote: Get libunac from: http://www.nongnu.org/unac/ (it uses iconv) Extract the archive, compile it (make). Move pg_unac.so to your

Re: [HACKERS] unaccent

2002-09-18 Thread nngodinh
Sep 2002 17:04:56 +0300 (GMT) From: Oleg Bartunov [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: [HACKERS] unaccent On Wed, 18 Sep 2002 [EMAIL PROTECTED] wrote: The best way to use it is quite simple. If you want to index the table titles and title is the field