Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-04-09 Thread Jan Urbański
Peter Eisentraut writes: > On 2/12/15 7:28 AM, Jan Urbański wrote: >> +#if OPENSSL_VERSION_NUMBER < 0x1000 >> +/* OpenSSL 1.0.0 deprecates the CRYPTO_set_id_callback function and >> provides a >> + * default implementation, so there's no need for our

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-04-08 Thread Jan Urbański
Peter Eisentraut writes: > On 4/3/15 7:44 AM, Jan Urbański wrote: >> To reiterate: I recognise that not handling the callbacks is not the right >> answer. But not stomping on callbacks others might have set sounds like a >> minimal and safe improvement. > > I think yo

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-04-03 Thread Jan Urbański
Peter Eisentraut writes: > On 4/2/15 4:32 AM, Jan Urbański wrote: >> >> Peter Eisentraut writes: >>> I don't think this patch would actually fix the problem that was >>> described after the original bug report >>> (http://www.postgresql.org/mess

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-04-02 Thread Jan Urbański
Peter Eisentraut writes: > On 2/12/15 7:28 AM, Jan Urbański wrote: >> For the sake of discussion, here's a patch to prevent stomping on >> previously-set callbacks, racy as it looks. >> >> FWIW, it does fix the Python deadlock and doesn't cause the PHP seg

Re: [HACKERS] GSoC idea - Simulated annealing to search for query plans

2015-02-27 Thread Jan Urbański
gt;> > I seem to recall somebody demo'ing a simulated-annealing GEQO >> replacement >>> > at PGCon a couple years back. It never got to the point of being a >>> > submitted patch though. >>> >>> Yea, it was Jan Urbański (CCed). >>

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-02-12 Thread Jan Urbański
Jan Urbański writes: > Andres Freund writes: > >> On 2015-02-12 09:31:27 +0100, Jan Urbański wrote: >>> That doesn't solve the problem of the Python deadlock, where you're not at >>> leisure to call a C function at the beginning of your module. &g

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-02-12 Thread Jan Urbański
Andres Freund writes: > On 2015-02-12 09:31:27 +0100, Jan Urbański wrote: >> That doesn't solve the problem of the Python deadlock, where you're not at >> leisure to call a C function at the beginning of your module. > > We could just never unload the hook

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-02-12 Thread Jan Urbański
Andres Freund writes: > On 2015-02-09 18:17:14 +0100, Jan Urbański wrote: >> First of all, the current behaviour is crazy. We're setting and unsetting the >> locking callback every time a connection is made/closed, which is not how >> OpenSSL is supposed to be used. The

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-02-11 Thread Jan Urbański
Jan Urbański writes: > I did some more digging on bug > http://www.postgresql.org/message-id/cahul3dpwyfnugdgo95ohydq4kugdnbkptjq0mnbtubhcmg4...@mail.gmail.com > which describes a deadlock when using libpq with SSL in a multi-threaded > environment with other threads doing SSL i

[HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-02-09 Thread Jan Urbański
I did some more digging on bug http://www.postgresql.org/message-id/cahul3dpwyfnugdgo95ohydq4kugdnbkptjq0mnbtubhcmg4...@mail.gmail.com which describes a deadlock when using libpq with SSL in a multi-threaded environment with other threads doing SSL independently. Attached is a reproducing Python s

Re: [HACKERS] Min value for port

2013-06-27 Thread Jan Urbański
On 27/06/13 15:11, Magnus Hagander wrote: On Thu, Jun 27, 2013 at 2:16 PM, Peter Eisentraut wrote: On 6/27/13 6:34 AM, Magnus Hagander wrote: Is there a reason why we have set the min allowed value for port to 1, not 1024? Given that you can't actually start postgres with a value of <1024, sho

Re: [HACKERS] updated emacs configuration

2013-06-26 Thread Jan Urbański
On 26/06/13 10:51, Dimitri Fontaine wrote: Peter Eisentraut writes: $ git clone git://git.postgresql.org/git/postgresql.git Cloning into 'postgresql'... I can reproduce that here. I don't know why I have those postgres dirs then, and I'm pretty confused about my round of testing now. Maybe

Re: [HACKERS] storing plpython global pointer

2013-05-28 Thread Jan Urbański
On 28/05/13 14:04, Szymon Guz wrote: Hi, I need to store a global pointer for plpython usage. This is a PyObject* which can be initialized per session I think Where should I keep such a pointer? Hi, you probably could use a global variable, similar to PLy_interp_globals that's defined in plp

Re: [HACKERS] plpgsql versus SPI plan abstraction

2013-01-30 Thread Jan Urbański
On 30/01/13 22:23, Tom Lane wrote: BTW, I'm also wondering if it's really necessary for plpython/plpy_spi.c to be looking into spi_priv.h ... As far as I can tell, it's not necessary, spi.h would be perfectly fine. Cheers, Jan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.

Re: [HACKERS] pg_dump --pretty-print-views

2013-01-28 Thread Jan Urbański
On 28/01/13 12:31, Marko Tiikkaja wrote: On 1/28/13 12:14 PM, Jeevan Chalke wrote: I could not apply the patch with git apply, but able to apply it by patch -p1 command. IME that's normal for patches that went through filterdiff. I do: git diff |filterdiff --format=context to re-format the p

[HACKERS] fix SQL example syntax in file comment

2013-01-12 Thread Jan Urbański
Hi, Here's a trivial patch that fixes a comment in execProcNode.c For archeological interest, that comment dates back to when it was written in POSTQUEL... The cleanup in a9b1ff4c1d699c8aa615397d47bb3071275c64ef changed RETRIEVE to SELECT, but forgot to add a FROM clause :) Cheers, Jan diff

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2013-01-09 Thread Jan Urbański
On 12/12/12 20:21, Karl O. Pinc wrote: There were 2 outstanding issue raised: Is this useful enough/proceeding in the right direction to commit now? Should some of the logic be moved out of a subroutine and into the calling code? I can see arguments to be made for both sides. I'm asking

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2012-12-12 Thread Jan Urbański
Though not the original patch autor, I did modify and submit it to the CF app, so I'll reply here :) On 10/12/12 19:20, Karl O. Pinc wrote: On 12/09/2012 10:33:59 PM, Karl O. Pinc wrote: I've gone ahead and signed up to review this patch. Thanks! There were 2 outstanding issue raised: I

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2012-11-05 Thread Jan Urbański
On 05/11/12 19:07, Jan Urbański wrote: On 05/11/12 18:35, Robert Haas wrote: You should probably add this to the next CF so we don't forget about it. I will, as soon as I recover my community account. Added as https://commitfest.postgresql.org/action/patch_view?id=971 J -- Sen

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2012-11-05 Thread Jan Urbański
On 05/11/12 18:35, Robert Haas wrote: On Wed, Oct 31, 2012 at 5:33 AM, Jan Urbański wrote: On 30/10/12 22:06, Oskari Saarenmaa wrote: PL/Python maps Python SPIError exceptions with 'spidata' attribute into SQL errors. Here's an alternative patch that takes advantage of the

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2012-10-31 Thread Jan Urbański
On 30/10/12 22:06, Oskari Saarenmaa wrote: PL/Python maps Python SPIError exceptions with 'spidata' attribute into SQL errors. PL/Python also creates classes in plpy.spiexceptions for all known errors but does not initialize their spidata, so when a PL/Python function raises such an exception it

[HACKERS] trivial typo in src/tools/RELEASE_CHANGES

2012-09-23 Thread Jan Urbański
There's a typo in src/tools/RELEASE_CHANGES It just ticked off my OCD I guess... Cheers, Jan diff --git a/src/tools/RELEASE_CHANGES b/src/tools/RELEASE_CHANGES new file mode 100644 index 5f1277a..aba1630 *** a/src/tools/RELEASE_CHANGES --- b/src/tools/RELEASE_CHANGES *** Then doing i

Re: [HACKERS] PostgreSQL 9.2beta4 (& git HEAD) server crash on creating extension plpython3u

2012-08-22 Thread Jan Urbański
On 22/08/12 13:28, Sachin Srivastava wrote: Yes, It worked for me also.. So will this be a workaround? Or do we intend to use something like Py_SetPythonHome() before calling Py_Initialize()/ I think the best we can do is to document that for some installations you might need to set PYTHONHOME

Re: [HACKERS] PostgreSQL 9.2beta4 (& git HEAD) server crash on creating extension plpython3u

2012-08-22 Thread Jan Urbański
On 21/08/12 20:13, Josh Berkus wrote: No. I get the same backtrace when I try against the 9.1.5 (REL9_1_STABLE) branch. I have reproduced this on Linux, seems like the fix is to to run the postmaster with this env variable exported: PYTHONHOME=/opt/ActivePython-3.2/ (or wherever you insta

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-08-09 Thread Jan Urbański
On 06/08/12 13:59, Heikki Linnakangas wrote: On 20.07.2012 10:13, Jan Urbański wrote: On 20/07/12 08:59, Jan Urbański wrote: On 18/07/12 17:17, Heikki Linnakangas wrote: On 14.07.2012 17:50, Jan Urbański wrote: If pg_do_encoding_conversion() throws an error, you don't get a chance to

Re: [HACKERS] Passing tabular data around using python functions

2012-07-31 Thread Jan Urbański
On 30/07/12 14:33, Achim Domma wrote: Hi, Hi Achim, this list is meant for discussing the development of PostgreSQL, in the future you might want to ask your question on pgsql-general. However, to answer your question: I call the function like this: select * from vectormatch(array(select

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-20 Thread Jan Urbański
On 20/07/12 08:59, Jan Urbański wrote: On 18/07/12 17:17, Heikki Linnakangas wrote: On 14.07.2012 17:50, Jan Urbański wrote: If pg_do_encoding_conversion() throws an error, you don't get a chance to call Py_DECREF() to release the string. Is that a problem? If an error occurs in PLy_trac

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-20 Thread Jan Urbański
On 18/07/12 17:17, Heikki Linnakangas wrote: On 14.07.2012 17:50, Jan Urbański wrote: If pg_do_encoding_conversion() throws an error, you don't get a chance to call Py_DECREF() to release the string. Is that a problem? If an error occurs in PLy_traceback(), after incrementing recursion_

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-14 Thread Jan Urbański
On 13/07/12 13:38, Jan Urbański wrote: On 12/07/12 11:08, Heikki Linnakangas wrote: On 07.07.2012 00:12, Jan Urbański wrote: So you're in favour of doing unicode -> bytes by encoding with UTF-8 and then using the server's encoding functions? Sounds reasonable to me. The ext

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-13 Thread Jan Urbański
On 12/07/12 11:08, Heikki Linnakangas wrote: On 07.07.2012 00:12, Jan Urbański wrote: On 06/07/12 22:47, Peter Eisentraut wrote: On fre, 2012-07-06 at 18:53 +0300, Heikki Linnakangas wrote: What shall we do about those? Ignore them? Document that if you're sing one of these encodings th

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-06 Thread Jan Urbański
On 06/07/12 22:47, Peter Eisentraut wrote: On fre, 2012-07-06 at 18:53 +0300, Heikki Linnakangas wrote: What shall we do about those? Ignore them? Document that if you're sing one of these encodings then PL/Python with Python 2 will be crippled and with Python 3 just won't work? We could conve

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-06 Thread Jan Urbański
On 06/07/12 10:14, Jan Urbański wrote: On 06/07/12 10:05, Heikki Linnakangas wrote: In the backend elog routines, there is a global variable 'recursion_depth', which is incremented when an error-handling routine is entered, and decremented afterwards. Can we use a similar mechinism i

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-06 Thread Jan Urbański
On 06/07/12 10:05, Heikki Linnakangas wrote: On 06.07.2012 00:54, Jan Urbański wrote: On 05/07/12 23:30, Peter Eisentraut wrote: On tor, 2012-07-05 at 22:53 +0200, Jan Urbański wrote: The problem is that PLyUnicode_Bytes is (via an ifdef) used as PyString_ToString on Python3, which means that

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-05 Thread Jan Urbański
On 05/07/12 23:30, Peter Eisentraut wrote: On tor, 2012-07-05 at 22:53 +0200, Jan Urbański wrote: The problem is that PLyUnicode_Bytes is (via an ifdef) used as PyString_ToString on Python3, which means that there are numerous call sites and new ones might appear in any moment. I'm not

[HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-05 Thread Jan Urbański
On 05/07/12 22:37, Heikki Linnakangas wrote: On 05.07.2012 23:31, Tom Lane wrote: Heikki Linnakangas writes: Fix mapping of PostgreSQL encodings to Python encodings. The buildfarm doesn't like this --- did you check for side effects on regression test results? Hmm, I ran the regressions tes

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-05 Thread Jan Urbański
On 05/07/12 21:37, Heikki Linnakangas wrote: Committed. This bug was present in versions >= 9.0, so backpatched. Thanks! I used ereport() rather than elog() in the error message. Correct me if that was wrong, but the point was to avoid PLy_elog(), because that might cause recursion, and erepo

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-04 Thread Jan Urbański
On 04/07/12 13:58, Asif Naeem wrote: Patch attached. Asif, could you try a few things on a CP1252 database? First verify if your original test case now works and then try this: I have test the patch on Win64. postgres server is working fine now for WIN1252. Thanks. create function enctes

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-03 Thread Jan Urbański
On 03/07/12 17:45, Jan Urbański wrote: On 29/06/12 00:36, Jan Urbański wrote: On 27/06/12 13:57, Jan Urbański wrote: On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. So: I'd add code to translate WINxxx

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-03 Thread Jan Urbański
On 29/06/12 00:36, Jan Urbański wrote: On 27/06/12 13:57, Jan Urbański wrote: On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. So: I'd add code to translate WINxxx into CPxxx when choosing the Python t

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-06-28 Thread Jan Urbański
On 27/06/12 13:57, Jan Urbański wrote: On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. CREATE PROCEDURAL LANGUAGE 'plpython3u'; CREATE OR REPLACE FUNCTION pymax (a integer, b integer) RETURNS integer

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-06-27 Thread Jan Urbański
On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. CREATE PROCEDURAL LANGUAGE 'plpython3u'; CREATE OR REPLACE FUNCTION pymax (a integer, b integer) RETURNS integer AS $$ if a> b: return a return b $$ LA

Re: [HACKERS] PL/Python result set slicing broken in Python 3

2012-05-11 Thread Jan Urbański
On 10/05/12 19:45, Peter Eisentraut wrote: On lör, 2012-05-05 at 22:45 +0200, Jan Urbański wrote: Apparently once you implement PyMappingMethods.mp_subscript you can drop PySequenceMethods.sq_slice, but I guess there's no harm in keeping it (and I'm not sure it'd work on Pytho

Re: [HACKERS] PL/Python result set slicing broken in Python 3

2012-05-05 Thread Jan Urbański
On 04/05/12 20:00, Jan Urbański wrote: On 03/05/12 11:04, Jan Urbański wrote: On 02/05/12 20:18, Peter Eisentraut wrote: This doesn't work anymore with Python 3: rv = plpy.execute(...) do_something(rv[0:1]) Sounds ugly. I'll take a look. I found some instructions on how to dea

Re: [HACKERS] PL/Python result set slicing broken in Python 3

2012-05-04 Thread Jan Urbański
On 03/05/12 11:04, Jan Urbański wrote: On 02/05/12 20:18, Peter Eisentraut wrote: This doesn't work anymore with Python 3: rv = plpy.execute(...) do_something(rv[0:1]) Apparently, they changed the C API for doing slicing, or rather made one of the two APIs for it silently do nothing. De

Re: [HACKERS] PL/Python result set slicing broken in Python 3

2012-05-03 Thread Jan Urbański
On 02/05/12 20:18, Peter Eisentraut wrote: This doesn't work anymore with Python 3: rv = plpy.execute(...) do_something(rv[0:1]) Apparently, they changed the C API for doing slicing, or rather made one of the two APIs for it silently do nothing. Details are difficult to find, but this email me

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-22 Thread Jan Urbański
On 10/04/12 21:47, Jan Urbański wrote: On 10/04/12 21:27, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: Yes, that would be ideal, even though not backwards-compatible. Back-patching is out of the question, but do we want to change trigger functions to receive dictionaries in NEW

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Jan Urbański
On 10/04/12 21:27, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: Yes, that would be ideal, even though not backwards-compatible. Back-patching is out of the question, but do we want to change trigger functions to receive dictionaries in NEW? Hm, I was not thinking of this as being

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Jan Urbański
On 10/04/12 20:47, Tom Lane wrote: I wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: Now that I understand what's been going on, I'll try to think of a non-invasive way of fixing that... ISTM that conversion of a composite value to Python ought to produce a dict, now that the other directio

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Jan Urbański
On 10/04/12 07:35, Jan Urbański wrote: On 10/04/12 04:20, Tom Lane wrote: Don't know if anybody noticed bug #6559 http://archives.postgresql.org/pgsql-bugs/2012-03/msg00180.php I've confirmed that the given test case works in 9.0 but fails in 9.1 and HEAD. So, I know what's go

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-09 Thread Jan Urbański
On 10/04/12 04:20, Tom Lane wrote: Don't know if anybody noticed bug #6559 http://archives.postgresql.org/pgsql-bugs/2012-03/msg00180.php I've confirmed that the given test case works in 9.0 but fails in 9.1 and HEAD. I find this code pretty unreadable, though, and know nothing to speak of abou

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-22 Thread Jan Urbański
On 21/02/12 18:28, Jan Urbański wrote: > On 21/02/12 18:05, Peter Eisentraut wrote: >>> it might be better to use ereport, to expose the message >>> for translation. >>> > After giving it some thought some of these elogs could be changed into > PLy_elogs (which

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-21 Thread Jan Urbański
On 21/02/12 18:05, Peter Eisentraut wrote: > On sön, 2012-02-19 at 22:29 -0500, Tom Lane wrote: >> My only comment is whether elog(ERROR) is appropriate, ie, do we >> consider these to be internal errors that users will never see in >> practice? If there's a significant risk of the error being thro

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-20 Thread Jan Urbański
On 20/02/12 04:29, Tom Lane wrote: > =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >>> On 18/02/12 21:17, Tom Lane wrote: Dave Malcolm at Red Hat has been working on a static code analysis tool for Python-related C code. He reports here on some preliminary results for plpython.c: h

Re: [HACKERS] pl/python long-lived allocations in datum->dict transformation

2012-02-19 Thread Jan Urbański
On 14/02/12 01:35, Tom Lane wrote: > =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >> It's not very comfortable, but >> I think PLyDict_FromTuple can be allowed to be non-reentrant. > > I think that's pretty short-sighted. Even if it's safe today (which > I am not 100% convinced of), there are plenty

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-19 Thread Jan Urbański
On 18/02/12 21:18, Jan Urbański wrote: > On 18/02/12 21:17, Tom Lane wrote: >> =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >>> On 18/02/12 20:30, Tom Lane wrote: >>>> Dave Malcolm at Red Hat has been working on a static code analysis tool >>>> for Pytho

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-18 Thread Jan Urbański
On 18/02/12 21:17, Tom Lane wrote: > =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >> On 18/02/12 20:30, Tom Lane wrote: >>> Dave Malcolm at Red Hat has been working on a static code analysis tool >>> for Python-related C code. He reports here on some preliminary results >>> for plpython.c: >>> https:

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-18 Thread Jan Urbański
On 18/02/12 20:30, Tom Lane wrote: > Dave Malcolm at Red Hat has been working on a static code analysis tool > for Python-related C code. He reports here on some preliminary results > for plpython.c: > https://bugzilla.redhat.com/show_bug.cgi?id=795011 > > I'm not enough of a Python hacker to eva

Re: [HACKERS] pl/python long-lived allocations in datum->dict transformation

2012-02-13 Thread Jan Urbański
On 12/02/12 00:48, Tom Lane wrote: > =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >> This is annoying for functions that plough through large tables, doing >> some calculation. Attached is a patch that does the conversion of >> PostgreSQL Datums into Python dict objects in a scratch memory context >>

[HACKERS] plpgsql leaking memory when stringifying datums

2012-02-05 Thread Jan Urbański
While chasing a PL/Python memory leak, I did a few tests with PL/pgSQL and I think there are places where memory is not freed sufficiently early. Attached are two functions that when run will make the backend's memory consumption increase until they finish. With both, the cause is convert_value_to

[HACKERS] pl/python long-lived allocations in datum->dict transformation

2012-02-05 Thread Jan Urbański
Consider this: create table arrays as select array[random(), random(), random(), random(), random(), random()] as a from generate_series(1, 100); create or replace function plpython_outputfunc() returns void as $$ c = plpy.cursor('select a from arrays') for row in c: pass $$ language plpy

[HACKERS] unfriendly error when accessing NEW in statement-level trigger

2012-01-28 Thread Jan Urbański
When you try to use the NEW variable in a statement-level trigger, you get ERROR: record "new" is not assigned yet DETAIL: The tuple structure of a not-yet-assigned record is indeterminate. which is not too friendly, given that people sometimes forget to specify FOR EACH at all, get the defaul

Re: [HACKERS] PL/Python result metadata

2012-01-11 Thread Jan Urbański
On 11/01/12 22:52, Dimitri Fontaine wrote: > Peter Eisentraut writes: >> .colnames() returns a list of column names (strings) >> .coltypes() returns a list of type OIDs (integers) >> >> I just made that up because there is no guidance in the other standard >> PLs for this sort of thing, AFAICT. >

Re: [HACKERS] pgsphere

2012-01-07 Thread Jan Urbański
- Original message - > On Saturday, January 07, 2012 04:43:43 PM Dave Cramer wrote: > > Well I've sent Teodor a personal email asking him if he was interested > > and so far no response, so I interpret that as he no longer has > > interest in the project. > I dimly remember him mentioning t

Re: [HACKERS] splitting plpython into smaller parts

2011-12-18 Thread Jan Urbański
On 18/12/11 20:53, Peter Eisentraut wrote: > On tis, 2011-12-06 at 00:58 +0100, Jan Urbański wrote: >> Rebased against master after the SPI cursor patch has been committed. >> >> The first patch removes SPI boilerplate from the cursor functions as >> well and

Re: [HACKERS] JSON for PG 9.2

2011-12-18 Thread Jan Urbański
On 18/12/11 04:21, Robert Haas wrote: > On Sat, Dec 17, 2011 at 7:50 PM, David E. Wheeler > wrote: >> Love having the start here. I forwarded this message to Claes Jakobsson, >> creator of the jansson-using pg-json extension. He’s a bit less supportive. >> He gave me permission to quote him her

Re: [HACKERS] JSON for PG 9.2

2011-12-12 Thread Jan Urbański
- Original message - > On Dec 12, 2011, at 4:51 PM, Peter van Hardenberg wrote: > > > Because we haven't heard from him in a while we've been using PL/V8 to > > validate a JSON datatype simulated by a DOMAIN with a simple > > acceptance function. (See below.) This is not ideally performant

Re: [HACKERS] [COMMITTERS] pgsql: plpython: Add SPI cursor support

2011-12-07 Thread Jan Urbański
On 07/12/11 11:16, Jan Urbański wrote: > On 06/12/11 19:33, Jan Urbański wrote: >> On 06/12/11 19:23, Tom Lane wrote: >>> Peter Eisentraut writes: >>>> plpython: Add SPI cursor support >>> >>> Buildfarm member narwhal does not like this patch.

Re: [HACKERS] [COMMITTERS] pgsql: plpython: Add SPI cursor support

2011-12-07 Thread Jan Urbański
On 06/12/11 19:33, Jan Urbański wrote: > On 06/12/11 19:23, Tom Lane wrote: >> Peter Eisentraut writes: >>> plpython: Add SPI cursor support >> >> Buildfarm member narwhal does not like this patch. It looks like >> "PyObject_SelfIter" is not a com

Re: [HACKERS] [COMMITTERS] pgsql: plpython: Add SPI cursor support

2011-12-06 Thread Jan Urbański
On 06/12/11 19:23, Tom Lane wrote: > Peter Eisentraut writes: >> plpython: Add SPI cursor support > > Buildfarm member narwhal does not like this patch. It looks like > "PyObject_SelfIter" is not a compile-time constant on its version > of python (2.5, apparently). Hm, I quickly tried with a se

Re: [HACKERS] plpython SPI cursors

2011-12-05 Thread Jan Urbański
On 05/12/11 19:12, Bruce Momjian wrote: > Jan Urbański wrote: >> On 05/12/11 18:58, Peter Eisentraut wrote: >>> On ons, 2011-11-23 at 19:58 +0100, Jan Urba?ski wrote: >>>> On 20/11/11 19:14, Steve Singer wrote: >>>> Responding now to all questions and att

Re: [HACKERS] plpython SPI cursors

2011-12-05 Thread Jan Urbański
On 05/12/11 18:58, Peter Eisentraut wrote: > On ons, 2011-11-23 at 19:58 +0100, Jan Urbański wrote: >> On 20/11/11 19:14, Steve Singer wrote: >> Responding now to all questions and attaching a revised patch based on >> your comments. > > Committed > > Pleas

Re: [HACKERS] Java LISTEN/NOTIFY client library work-around

2011-11-30 Thread Jan Urbański
On 30/11/11 13:07, Joel Jacobson wrote: Hi, As you know, LISTEN/NOTIFY is broken in the Java client library. You have to do a SELECT 1 in a while-loop to receive the notifications. http://jdbc.postgresql.org/documentation/head/listennotify.html Is there some other library with a proper impleme

Re: [HACKERS] splitting plpython into smaller parts

2011-11-28 Thread Jan Urbański
On 28/11/11 11:00, Greg Smith wrote: On 11/13/2011 09:45 AM, Jan Urbański wrote: The second one is the actual split. plpython.c has been split into 11 separate files and one header. Could you comment a bit more about what the goal of this is? We don't have a reviewer for this patch yet,

Re: [HACKERS] PL/Python SQL error code pass-through

2011-11-24 Thread Jan Urbański
On 24/11/11 16:15, Heikki Linnakangas wrote: On 24.11.2011 10:07, Jan Urbański wrote: On 23/11/11 17:24, Mika Eloranta wrote: Hi all, [PL/Python in 9.1 does not preserve SQLSTATE of errors] Oops, you're right, it's a regression from 9.0 behaviour. The fix looks good to me, I c

Re: [HACKERS] PL/Python SQL error code pass-through

2011-11-24 Thread Jan Urbański
On 23/11/11 17:24, Mika Eloranta wrote: Hi all, [PL/Python in 9.1 does not preserve SQLSTATE of errors] Oops, you're right, it's a regression from 9.0 behaviour. The fix looks good to me, I changed one place to indent with tabs instead of spaces and added a regression test. I think this sh

Re: [HACKERS] plpython SPI cursors

2011-11-23 Thread Jan Urbański
On 20/11/11 19:14, Steve Singer wrote: On 11-10-15 07:28 PM, Jan Urbański wrote: Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. Currently when trying to process a large table in PL/Python you have slurp it all into memory (that's what plpy.execute does).

Re: [HACKERS] plpython SPI cursors

2011-11-20 Thread Jan Urbański
On 20/11/11 19:14, Steve Singer wrote: On 11-10-15 07:28 PM, Jan Urbański wrote: Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. I found a few bugs (see my testing section below) that will need fixing + a few questions about the code Hi Steve, thanks a lot for

Re: [HACKERS] Monster query

2011-10-24 Thread Jan Urbański
On 24/10/11 19:16, Omar Bettin wrote: > Hi, > > Since the data are of third parties, will prepare a database suitable for the > purpose. > In any case, the compressed backup will be around 20 MB. If you are able to prepare a database dump that doesn't contain private data, it would be awesome.

Re: [HACKERS] [9.1] unusable for large views

2011-10-24 Thread Jan Urbański
On 24/10/11 10:57, Omar Bettin wrote: > > [monster query] I see that your problem is already solved, but incidentially I'm working on a join order planning module and I'm looking for real-life examples of humongous queries like that to benchmark against them. Any chance you could share the schem

Re: [HACKERS] proposal: set GUC variables for single query

2011-10-17 Thread Jan Urbański
On 17/10/11 02:53, Robert Haas wrote: > On Sun, Oct 16, 2011 at 4:58 PM, Tom Lane wrote: >> Dimitri Fontaine writes: >>> Now that you mention it, the following might actually already work: >> >>> WITH settings AS ( >>>SELECT set_config('timezone', 'Europe/Amsterdam', t), >>> set_co

Re: [HACKERS] proposal: set GUC variables for single query

2011-10-16 Thread Jan Urbański
On 16/10/11 17:49, Tom Lane wrote: > =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >> this idea has cropped up last PGCon - the ability to set GUC variables >> for the duration of a single query. It would work by setting the GUCs >> for the duration of the query and setting them back to what they were

[HACKERS] proposal: set GUC variables for single query

2011-10-16 Thread Jan Urbański
Hi, this idea has cropped up last PGCon - the ability to set GUC variables for the duration of a single query. It would work by setting the GUCs for the duration of the query and setting them back to what they were after it has terminated. By "setting them back" I mean respecting the previously se

[HACKERS] plpython SPI cursors

2011-10-15 Thread Jan Urbański
Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. Currently when trying to process a large table in PL/Python you have slurp it all into memory (that's what plpy.execute does). This patch allows reading the result set in smaller chunks, using a SPI cursor behind the scen

Re: [HACKERS] Error building v9.1.1 (git) with python 3.2.2

2011-10-05 Thread Jan Urbański
On 05/10/11 15:24, Lou Picciano wrote: > Hackers, > > Have been following with some interest the dialog recently about getting > pl/python built within the v9 tree; most recently, I've been trying v9.1.1 > (git) and python 3.2.2. Configure and make complete without error, but 'make > install'

Re: [HACKERS] Large C files

2011-09-06 Thread Jan Urbański
On 07/09/11 01:13, Peter Geoghegan wrote: > On 6 September 2011 08:29, Peter Eisentraut wrote: >> I was thinking about splitting up plpython.c, but it's not even on that >> list. ;-) > > IIRC the obesity of that file is something that Jan Urbański intends > to fix, o

Re: [HACKERS] any results from PL summit?

2011-08-29 Thread Jan Urbański
On 29/08/11 08:21, Pavel Stehule wrote: > Hello Hi Pavel, > is there some some result, report from PL summit? The notes taken during the summit are here: http://wiki.postgresql.org/wiki/PgCon_2011_PL_Summit I think Selena also sent a "summary and next steps" mail to the participants, perhaps i

Re: [HACKERS] plpython crash

2011-08-18 Thread Jan Urbański
On 17/08/11 23:10, Tom Lane wrote: > =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >> Here are two patches that fix two separate bugs that you found >> simultaneously. Because they're actually separate issues, it turned out >> fixing them was a bit more tricky than I expected (fixing one was >> unmaski

Re: [HACKERS] rc1 or beta4?

2011-08-17 Thread Jan Urbański
On 17/08/11 17:50, Thom Brown wrote: > On 17 August 2011 16:47, Jan Urbański wrote: > >> On 17/08/11 15:00, Dave Page wrote: >>> The current plan (or, the last one I recall) is to push another 9.1 >>> release tomorrow, for Monday release. Are we going with beta4

Re: [HACKERS] rc1 or beta4?

2011-08-17 Thread Jan Urbański
On 17/08/11 15:00, Dave Page wrote: > The current plan (or, the last one I recall) is to push another 9.1 > release tomorrow, for Monday release. Are we going with beta4 or rc1? Sorry to butt in, but it would probably be good to include fixes for the two segfault plpython bugs[1] before wrapping u

Re: [HACKERS] plpython crash

2011-08-17 Thread Jan Urbański
On 17/08/11 11:40, Jan Urbański wrote: > On 16/08/11 19:12, Jan Urbański wrote: >> On 16/08/11 19:07, Jean-Baptiste Quenot wrote: >>> >>> [plpython is buggy] > I'll have a patch ready soon. Here are two patches that fix two separate bugs that you found simul

Re: [HACKERS] Caching Python modules

2011-08-17 Thread Jan Urbański
On 17/08/11 14:19, Jan Urbański wrote: > On 17/08/11 14:09, PostgreSQL - Hans-Jürgen Schönig wrote: >> CREATE OR REPLACE FUNCTION textprocess.add_to_corpus(lang text, t text) >> RETURNS float4 AS $$ >> >> from SecondCorpus import SecondCorpus >>

Re: [HACKERS] Caching Python modules

2011-08-17 Thread Jan Urbański
On 17/08/11 14:09, PostgreSQL - Hans-Jürgen Schönig wrote: > CREATE OR REPLACE FUNCTION textprocess.add_to_corpus(lang text, t text) > RETURNS float4 AS $$ > > from SecondCorpus import SecondCorpus > from SecondDocument import SecondDocument > > i am doing some intense text minin

Re: [HACKERS] plpython crash

2011-08-17 Thread Jan Urbański
On 16/08/11 19:12, Jan Urbański wrote: > On 16/08/11 19:07, Jean-Baptiste Quenot wrote: >> Dear Jan, >> >> Sorry I typed the wrong git commands. With latest plpython from >> branch master I got the same gdb backtrace as reported before. I >> managed to wrap up a

Re: [HACKERS] plpython crash

2011-08-16 Thread Jan Urbański
On 16/08/11 19:07, Jean-Baptiste Quenot wrote: > Dear Jan, > > Sorry I typed the wrong git commands. With latest plpython from > branch master I got the same gdb backtrace as reported before. I > managed to wrap up a testcase that fails 100% of times on my setup: > https://gist.github.com/114951

Re: [HACKERS] plpython crash

2011-08-16 Thread Jan Urbański
On 16/08/11 17:06, Jan Urbański wrote: > On 16/08/11 16:52, Jean-Baptiste Quenot wrote: >> After backporting plpython.c from HEAD, this is the error message I get: >> >> ERROR: key "pg.dropped.6" not found in mapping >> HINT: To return null i

Re: [HACKERS] plpython crash

2011-08-16 Thread Jan Urbański
On 16/08/11 16:52, Jean-Baptiste Quenot wrote: > After backporting plpython.c from HEAD, this is the error message I get: > > ERROR: key "pg.dropped.6" not found in mapping > HINT: To return null in a column, add the value None to the mapping > with the key named after the column

Re: [HACKERS] plpython crash

2011-08-12 Thread Jan Urbański
On 12/08/11 13:55, Jean-Baptiste Quenot wrote: > Here is the same with -O0: > > https://gist.github.com/1140005 > > sys.version reports this: > > INFO: 2.6.6 (r266:84292, Sep 15 2010, 16:41:53) > [GCC 4.4.5] I'm still at a loss. Did you reproduce it with git HEAD? I see that the query being ex

Re: [HACKERS] plpython crash

2011-08-12 Thread Jan Urbański
On 11/08/11 18:01, Jean-Baptiste Quenot wrote: > Hi there, > > plpython crashes on me on various 64-bit Ubuntu hosts, see the gdb > backtrace at: https://gist.github.com/1140005 > > Do you believe there was recent bugfixes regarding PLyMapping_ToTuple() ? > > This is PG 9.0.4 with HEAD of plpyth

Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-22 Thread Jan Urbański
On 23/07/11 01:12, Robert Haas wrote: > On Fri, Jul 22, 2011 at 6:04 PM, Joey Adams > wrote: >> On another matter, should the JSON type guard against duplicate member >> keys? The JSON RFC says "The names within an object SHOULD be >> unique," meaning JSON with duplicate members can be considere

Re: [HACKERS] PL/Python: No stack trace for an exception

2011-07-21 Thread Jan Urbański
On 21/07/11 15:27, Sushant Sinha wrote: > I am using plpythonu on postgres 9.0.2. One of my python functions was > throwing a TypeError exception. However, I only see the exception in the > database and not the stack trace. It becomes difficult to debug if the > stack trace is absent in Python. >

  1   2   3   4   >