Re: [HACKERS] PL/Python array support
On tis, 2009-12-01 at 20:53 -0700, Joshua Tolley wrote: This patch doesn't include any documentation; my reading of the PL/Python docs suggests that's probably acceptable, as the existing docs don't talk about its array handling. That said, it might be useful to include an example, to show for instance that identical PL/Python code could create either an array of a type or a set of rows of that type: I added a bit of documentation like that. Thanks. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PL/Python array support
On Fri, Nov 20, 2009 at 12:00:24AM +0200, Peter Eisentraut wrote:
On fre, 2009-11-13 at 18:46 +0300, Teodor Sigaev wrote:
CREATE OR REPLACE FUNCTION incr(stuff int[]) RETURNS int[] AS $$
for x in stuff:
yield x+1
$$
LANGUAGE 'plpythonu';
# select incr(ARRAY[1,2,3]);
ERROR: invalid memory alloc request size 18446744073709551608
CONTEXT: while creating return value
PL/Python function incr
Fixed with additional error check and regression test. (The problem
could be more simply demonstrated by returning any non-sequence from the
function.) Thanks for catching it.
My last email claimed that the regression test needed some additional changes
to its expected output, and further claimed that it had the regression test's
diff attached. As was helpfully pointed out off-list, it actually wasn't
attached. Trying again..
-- Josh
*** /home/josh/devel/pgsrc/pg85/src/pl/plpython/expected/plpython_types.out
2009-12-01 20:39:52.0 -0700
--- /home/josh/devel/pgsrc/pg85/src/pl/plpython/results/plpython_types.out
2009-12-01 20:40:04.0 -0700
***
*** 580,582
--- 580,589
{abc,def}
(1 row)
+ CREATE FUNCTION test_type_conversion_array_error() RETURNS int[] AS $$
+ return 5
+ $$ LANGUAGE plpythonu;
+ SELECT * FROM test_type_conversion_array_error();
+ ERROR: PL/Python: return value of function with array return type is not a
Python sequence
+ CONTEXT: while creating return value
+ PL/Python function test_type_conversion_array_error
==
signature.asc
Description: Digital signature
Re: [HACKERS] PL/Python array support
On Fri, Nov 20, 2009 at 12:00:24AM +0200, Peter Eisentraut wrote:
On fre, 2009-11-13 at 18:46 +0300, Teodor Sigaev wrote:
CREATE OR REPLACE FUNCTION incr(stuff int[]) RETURNS int[] AS $$
for x in stuff:
yield x+1
$$
LANGUAGE 'plpythonu';
# select incr(ARRAY[1,2,3]);
ERROR: invalid memory alloc request size 18446744073709551608
CONTEXT: while creating return value
PL/Python function incr
Fixed with additional error check and regression test. (The problem
could be more simply demonstrated by returning any non-sequence from the
function.) Thanks for catching it.
I've finally gotten around to getting a review done, and basically it looks
good to me. There appears to be a problem with the tests in that the expected
output doesn't include the test_type_conversion_array_error() function
mentioned in sql/plpython_types.sql. Diff generated by the regression test is
attached. Other than that problem, though, the code looks fine to me (should I
presume to judge Peter's code? :). Aside from the problem mentioned above, the
tests work fine, and seem fairly comprehensive. Other testing I've done also
passes.
This patch doesn't include any documentation; my reading of the PL/Python docs
suggests that's probably acceptable, as the existing docs don't talk about its
array handling. That said, it might be useful to include an example, to show
for instance that identical PL/Python code could create either an array of a
type or a set of rows of that type:
5432 j...@josh*# create function return_set() returns setof int as $$ return
(1, 2, 3, 4, 5) $$ language plpythonu;
CREATE FUNCTION
5432 j...@josh*# create function return_arr() returns int[] as $$ return (1,
2, 3, 4, 5) $$ language plpythonu;
CREATE FUNCTION
5432 j...@josh*# select return_arr();
return_arr
-
{1,2,3,4,5}
(1 row)
5432 j...@josh*# select * from return_set();
return_set
1
2
3
4
5
(5 rows)
Perhaps that's overkill, though.
--
Joshua Tolley / eggyknap
End Point Corporation
http://www.endpoint.com
signature.asc
Description: Digital signature
Re: [HACKERS] PL/Python array support
On fre, 2009-11-13 at 18:46 +0300, Teodor Sigaev wrote:
CREATE OR REPLACE FUNCTION incr(stuff int[]) RETURNS int[] AS $$
for x in stuff:
yield x+1
$$
LANGUAGE 'plpythonu';
# select incr(ARRAY[1,2,3]);
ERROR: invalid memory alloc request size 18446744073709551608
CONTEXT: while creating return value
PL/Python function incr
Fixed with additional error check and regression test. (The problem
could be more simply demonstrated by returning any non-sequence from the
function.) Thanks for catching it.
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index 2dd498c..cbc93a2 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -477,3 +477,106 @@ CONTEXT: PL/Python function test_type_conversion_bytea10
ERROR: value for domain bytea10 violates check constraint bytea10_check
CONTEXT: while creating return value
PL/Python function test_type_conversion_bytea10
+--
+-- Arrays
+--
+CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS int4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int4(ARRAY[0, 100]);
+INFO: ([0, 100], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {0,100}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[0,-100,55]);
+INFO: ([0, -100, 55], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {0,-100,55}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
+INFO: ([None, 1], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {NULL,1}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
+INFO: ([], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(NULL);
+INFO: (None, type 'NoneType')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+ERROR: cannot convert multidimensional array to Python list
+DETAIL: PL/Python only supports one-dimensional arrays.
+CONTEXT: PL/Python function test_type_conversion_array_int4
+CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_bytea(ARRAY[E'\\xdeadbeef'::bytea, NULL]);
+INFO: (['\xde\xad\xbe\xef', None], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_bytea
+ test_type_conversion_array_bytea
+--
+ {\\xdeadbeef,NULL}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_mixed1() RETURNS text[] AS $$
+return [123, 'abc']
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_mixed1();
+ test_type_conversion_array_mixed1
+---
+ {123,abc}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_mixed2() RETURNS int[] AS $$
+return [123, 'abc']
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_mixed2();
+ERROR: invalid input syntax for integer: abc
+CONTEXT: while creating return value
+PL/Python function test_type_conversion_array_mixed2
+CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
+return [None]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_record();
+ERROR: PL/Python functions cannot return type type_record[]
+DETAIL: PL/Python does not support conversion to arrays of row types.
+CREATE FUNCTION test_type_conversion_array_string() RETURNS text[] AS $$
+return 'abc'
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_string();
+ test_type_conversion_array_string
+---
+ {a,b,c}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_tuple() RETURNS text[] AS $$
+return ('abc', 'def')
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_tuple();
+ test_type_conversion_array_tuple
+--
+ {abc,def}
+(1 row)
+
diff --git a/src/pl/plpython/plpython.c b/src/pl/plpython/plpython.c
index 6fd4aca..6a2a12f 100644
--- a/src/pl/plpython/plpython.c
+++ b/src/pl/plpython/plpython.c
@@ -89,6 +89,9 @@ typedef struct PLyDatumToOb
Oid typoid; /* The OID of the type */
Oid typioparam;
bool typbyval;
+ int16 typlen;
+ char typalign;
+ struct PLyDatumToOb *elm;
} PLyDatumToOb;
typedef struct PLyTupleToOb
@@ -120,6 +123,9 @@ typedef struct PLyObToDatum
Oid typoid; /* The OID of the type */
Oid
Re: [HACKERS] PL/Python array support
CREATE OR REPLACE FUNCTION incr(stuff int[]) RETURNS int[] AS $$
for x in stuff:
yield x+1
$$
LANGUAGE 'plpythonu';
# select incr(ARRAY[1,2,3]);
ERROR: invalid memory alloc request size 18446744073709551608
CONTEXT: while creating return value
PL/Python function incr
Suppose, it could be fixed by additional check in PLy_function_handler near line
947 :
if (proc-is_setof) {
...
}
else if (PyIter_Check(plrv))
{
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg(returned object should be iterated),
errdetail(PL/Python returns iterable object in non-setof returning
context)));
}
Peter Eisentraut wrote:
On ons, 2009-11-04 at 16:02 +0200, Peter Eisentraut wrote:
Here is a patch to support arrays in PL/Python as parameters and
return values.
Slightly updated version with fixed reference counting.
--
Teodor Sigaev E-mail: teo...@sigaev.ru
WWW: http://www.sigaev.ru/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PL/Python array support
On ons, 2009-11-04 at 16:02 +0200, Peter Eisentraut wrote:
Here is a patch to support arrays in PL/Python as parameters and
return values.
Slightly updated version with fixed reference counting.
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index 2dd498c..cbc93a2 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -477,3 +477,106 @@ CONTEXT: PL/Python function test_type_conversion_bytea10
ERROR: value for domain bytea10 violates check constraint bytea10_check
CONTEXT: while creating return value
PL/Python function test_type_conversion_bytea10
+--
+-- Arrays
+--
+CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS int4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int4(ARRAY[0, 100]);
+INFO: ([0, 100], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {0,100}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[0,-100,55]);
+INFO: ([0, -100, 55], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {0,-100,55}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
+INFO: ([None, 1], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {NULL,1}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
+INFO: ([], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(NULL);
+INFO: (None, type 'NoneType')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+ERROR: cannot convert multidimensional array to Python list
+DETAIL: PL/Python only supports one-dimensional arrays.
+CONTEXT: PL/Python function test_type_conversion_array_int4
+CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_bytea(ARRAY[E'\\xdeadbeef'::bytea, NULL]);
+INFO: (['\xde\xad\xbe\xef', None], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_bytea
+ test_type_conversion_array_bytea
+--
+ {\\xdeadbeef,NULL}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_mixed1() RETURNS text[] AS $$
+return [123, 'abc']
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_mixed1();
+ test_type_conversion_array_mixed1
+---
+ {123,abc}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_mixed2() RETURNS int[] AS $$
+return [123, 'abc']
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_mixed2();
+ERROR: invalid input syntax for integer: abc
+CONTEXT: while creating return value
+PL/Python function test_type_conversion_array_mixed2
+CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
+return [None]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_record();
+ERROR: PL/Python functions cannot return type type_record[]
+DETAIL: PL/Python does not support conversion to arrays of row types.
+CREATE FUNCTION test_type_conversion_array_string() RETURNS text[] AS $$
+return 'abc'
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_string();
+ test_type_conversion_array_string
+---
+ {a,b,c}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_tuple() RETURNS text[] AS $$
+return ('abc', 'def')
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_tuple();
+ test_type_conversion_array_tuple
+--
+ {abc,def}
+(1 row)
+
diff --git a/src/pl/plpython/plpython.c b/src/pl/plpython/plpython.c
index 6fd4aca..8f097b3 100644
--- a/src/pl/plpython/plpython.c
+++ b/src/pl/plpython/plpython.c
@@ -89,6 +89,9 @@ typedef struct PLyDatumToOb
Oid typoid; /* The OID of the type */
Oid typioparam;
bool typbyval;
+ int16 typlen;
+ char typalign;
+ struct PLyDatumToOb *elm;
} PLyDatumToOb;
typedef struct PLyTupleToOb
@@ -120,6 +123,9 @@ typedef struct PLyObToDatum
Oid typoid; /* The OID of the type */
Oid typioparam;
bool typbyval;
+ int16 typlen;
+ char typalign;
+ struct PLyObToDatum *elm;
} PLyObToDatum;
typedef struct PLyObToTuple
@@ -284,6 +290,7 @@ static PyObject *PLyInt_FromInt32(PLyDatumToOb *arg, Datum d);
static PyObject *PLyLong_FromInt64(PLyDatumToOb *arg, Datum d);
static PyObject
[HACKERS] PL/Python array support
Here is a patch to support arrays in PL/Python as parameters and return
values. It converts an array parameter to a Python list, and converts
a Python sequence return value back to an array.
I have settled on two implementation restrictions for the moment:
- Only supports one-dimensional arrays. (Python has no multidimensional
lists, so the semantics of this would be dubious.)
- Does not support returning arrays of composite types. (Basically too
complicated to implement right now and seemingly of limited practical
value.)
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index 2dd498c..cbc93a2 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -477,3 +477,106 @@ CONTEXT: PL/Python function test_type_conversion_bytea10
ERROR: value for domain bytea10 violates check constraint bytea10_check
CONTEXT: while creating return value
PL/Python function test_type_conversion_bytea10
+--
+-- Arrays
+--
+CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS int4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int4(ARRAY[0, 100]);
+INFO: ([0, 100], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {0,100}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[0,-100,55]);
+INFO: ([0, -100, 55], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {0,-100,55}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
+INFO: ([None, 1], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {NULL,1}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
+INFO: ([], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+ {}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(NULL);
+INFO: (None, type 'NoneType')
+CONTEXT: PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4
+-
+
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+ERROR: cannot convert multidimensional array to Python list
+DETAIL: PL/Python only supports one-dimensional arrays.
+CONTEXT: PL/Python function test_type_conversion_array_int4
+CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_bytea(ARRAY[E'\\xdeadbeef'::bytea, NULL]);
+INFO: (['\xde\xad\xbe\xef', None], type 'list')
+CONTEXT: PL/Python function test_type_conversion_array_bytea
+ test_type_conversion_array_bytea
+--
+ {\\xdeadbeef,NULL}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_mixed1() RETURNS text[] AS $$
+return [123, 'abc']
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_mixed1();
+ test_type_conversion_array_mixed1
+---
+ {123,abc}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_mixed2() RETURNS int[] AS $$
+return [123, 'abc']
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_mixed2();
+ERROR: invalid input syntax for integer: abc
+CONTEXT: while creating return value
+PL/Python function test_type_conversion_array_mixed2
+CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
+return [None]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_record();
+ERROR: PL/Python functions cannot return type type_record[]
+DETAIL: PL/Python does not support conversion to arrays of row types.
+CREATE FUNCTION test_type_conversion_array_string() RETURNS text[] AS $$
+return 'abc'
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_string();
+ test_type_conversion_array_string
+---
+ {a,b,c}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_tuple() RETURNS text[] AS $$
+return ('abc', 'def')
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_tuple();
+ test_type_conversion_array_tuple
+--
+ {abc,def}
+(1 row)
+
diff --git a/src/pl/plpython/plpython.c b/src/pl/plpython/plpython.c
index 6fd4aca..8640496 100644
--- a/src/pl/plpython/plpython.c
+++ b/src/pl/plpython/plpython.c
@@ -89,6 +89,9 @@ typedef struct PLyDatumToOb
Oid typoid; /* The OID of the type */
Oid typioparam;
bool typbyval;
+ int16 typlen;
+ char typalign;
+ struct PLyDatumToOb *elm;
} PLyDatumToOb;
typedef struct PLyTupleToOb
@@ -120,6 +123,9 @@ typedef struct PLyObToDatum
Oid typoid; /* The OID of the type */
Re: [HACKERS] PL/Python array support
On Wed, Nov 4, 2009 at 9:02 AM, Peter Eisentraut pete...@gmx.net wrote: Here is a patch to support arrays in PL/Python as parameters and return values. It converts an array parameter to a Python list, and converts a Python sequence return value back to an array. This is probably a stupid question, but why would you use different types for incoming and outgoing data flow? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PL/Python array support
On ons, 2009-11-04 at 09:44 -0500, Robert Haas wrote: On Wed, Nov 4, 2009 at 9:02 AM, Peter Eisentraut pete...@gmx.net wrote: Here is a patch to support arrays in PL/Python as parameters and return values. It converts an array parameter to a Python list, and converts a Python sequence return value back to an array. This is probably a stupid question, but why would you use different types for incoming and outgoing data flow? A list is one particular kind of sequence. See also http://docs.python.org/reference/datamodel.html#the-standard-type-hierarchy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
