Re: [HACKERS] nested hstore patch - FailedAssertion(!(value-array.nelems == 1)
https://github.com/feodor/postgres On Wed, Jan 15, 2014 at 11:10 AM, Erik Rijkers e...@xs4all.nl wrote: On Wed, January 15, 2014 08:01, Oleg Bartunov wrote: It doesn't crashed in the last version in our repository. =# select 'x'::hstore || ('a=1':: hstore) ; ?column? --- x, a, 1 (1 row) OK, shall I use that repository instead of the latest posted patch? No point in testing old code ( I used nested_hstore_and_jsonb-2.patch ). Could you send a link to where that repository is? ( btw, your query is not quite the same as the one I used: select 'x' || ('a=1':: hstore) but your query also crashes my server here so I suppose it triggers the same bug ) thanks, Erik Rijkers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch - FailedAssertion(!(value-array.nelems == 1)
On Wed, January 15, 2014 09:46, Oleg Bartunov wrote: On Wed, January 15, 2014 08:01, Oleg Bartunov wrote: It doesn't crashed in the last version in our repository. =# select 'x'::hstore || ('a=1':: hstore) ; ?column? --- x, a, 1 (1 row) OK, shall I use that repository instead of the latest posted patch? I now installed from: https://github.com/feodor/postgres and compiled both a 'fast' and a 'debug' server (=with --enable-cassert see [1]) It turns out that the statement does not crash on a server compiled without --enable-cassert. But a compile with --enable-cassert shows that a bug is still lurking: testdb=# select 'x'::hstore || ('a=1':: hstore) ; The connection to the server was lost. Attempting reset: Failed. ! TRAP: FailedAssertion(!(value-array.nelems == 1), File: hstore_support.c, Line: 896) Not good. ( please note that the assert is in a different file ('hstore_support.c') from the earlier assert error that I posted ) Thanks, Erik Rijkers [1] pg_config: '--prefix=/home/aardvark/pg_stuff/pg_installations/pgsql.nested_hstore_url' '--bindir=/home/aardvark/pg_stuff/pg_installations/pgsql.nested_hstore_url/bin' '--libdir=/home/aardvark/pg_stuff/pg_installations/pgsql.nested_hstore_url/lib' '--with-pgport=46541' '--enable-depend' '--enable-cassert' '--enable-debug' '--with-openssl' '--with-perl' '--with-libxml' '--with-libxslt' '--with-zlib' -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
Erik, thanks for docs fixes, we have even more :) Oleg On Tue, Jan 14, 2014 at 4:18 AM, Erik Rijkers e...@xs4all.nl wrote: On Mon, January 13, 2014 18:30, Andrew Dunstan wrote: On 01/13/2014 11:16 AM, Oleg Bartunov wrote: Andrew, did you run perl script ? Actually, I found, that operator table needs to be fixed. No. My build machine doesn't actually have DBD::Pg installed. Can you send me a patch if you don't want to push it yourself, or maybe Erik can send a pacth top adjust the table. [ nested_hstore_and_jsonb-2.patch ] ( centos 6.5, gcc 4.8.2. ) The patch applies compiles with warnings (see below). The opr_sanity test fails during make check: regression.diffs attached. Also attached are changes to hstore.sgml, to operator + functions table, plus some typos. Thanks, Erik Rijkers make jsonfuncs.c: In function ‘each_object_field_end_jsonb’: jsonfuncs.c:1328:7: warning: assignment from incompatible pointer type [enabled by default] val = DatumGetPointer(DirectFunctionCall1(jsonb_in, CStringGetDatum(cstr))); ^ jsonfuncs.c: In function ‘elements_array_element_end_jsonb’: jsonfuncs.c:1530:8: warning: assignment from incompatible pointer type [enabled by default] jbval = DatumGetPointer(DirectFunctionCall1(jsonb_in, CStringGetDatum(cstr))); ^ make contrib: hstore_io.c: In function ‘array_to_hstore’: hstore_io.c:1694:30: warning: ‘result’ may be used uninitialized in this function [-Wmaybe-uninitialized] PG_RETURN_POINTER(hstoreDump(result)); -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch - FailedAssertion(!(value-array.nelems == 1)
On Mon, January 13, 2014 16:36, Andrew Dunstan wrote: A new version of the patch is attached. It includes all of Erik's docs [ nested_hstore_and_jsonb-2.patch ] This crashes the server: testdb=# select 'x' || ('a=1':: hstore) ; The connection to the server was lost. Attempting reset: Failed. logging: TRAP: FailedAssertion(!(value-array.nelems == 1), File: jsonb_support.c, Line: 904) 2014-01-15 00:32:01.854 CET 1206 LOG: server process (PID 3918) was terminated by signal 6: Aborted 2014-01-15 00:32:01.854 CET 1206 DETAIL: Failed process was running: select 'x' || ('a=1':: hstore) ; Btw, I find it strange that: testdb=# select ('a=':: hstore) #% '{a}' ; ?column? -- (1 row) so that: Time: 0.641 ms testdb=# select ( ('a=':: hstore) #% '{a}' ) = '' ; ?column? -- f (1 row) testdb=# select ( ('a=':: hstore) #% '{a}' ) = '' ; ?column? -- t (1 row) Maybe there is a rationale, but it seems to me that ('a=':: hstore) #% '{a}' should deliver the empty string '', and not two double quotes. Thanks, Erik Rijkers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch - FailedAssertion(!(value-array.nelems == 1)
It doesn't crashed in the last version in our repository. =# select 'x'::hstore || ('a=1':: hstore) ; ?column? --- x, a, 1 (1 row) On Wed, Jan 15, 2014 at 3:53 AM, Erik Rijkers e...@xs4all.nl wrote: On Mon, January 13, 2014 16:36, Andrew Dunstan wrote: A new version of the patch is attached. It includes all of Erik's docs [ nested_hstore_and_jsonb-2.patch ] This crashes the server: testdb=# select 'x' || ('a=1':: hstore) ; The connection to the server was lost. Attempting reset: Failed. logging: TRAP: FailedAssertion(!(value-array.nelems == 1), File: jsonb_support.c, Line: 904) 2014-01-15 00:32:01.854 CET 1206 LOG: server process (PID 3918) was terminated by signal 6: Aborted 2014-01-15 00:32:01.854 CET 1206 DETAIL: Failed process was running: select 'x' || ('a=1':: hstore) ; Btw, I find it strange that: testdb=# select ('a=':: hstore) #% '{a}' ; ?column? -- (1 row) so that: Time: 0.641 ms testdb=# select ( ('a=':: hstore) #% '{a}' ) = '' ; ?column? -- f (1 row) testdb=# select ( ('a=':: hstore) #% '{a}' ) = '' ; ?column? -- t (1 row) Maybe there is a rationale, but it seems to me that ('a=':: hstore) #% '{a}' should deliver the empty string '', and not two double quotes. Thanks, Erik Rijkers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch - FailedAssertion(!(value-array.nelems == 1)
On Wed, January 15, 2014 08:01, Oleg Bartunov wrote: It doesn't crashed in the last version in our repository. =# select 'x'::hstore || ('a=1':: hstore) ; ?column? --- x, a, 1 (1 row) OK, shall I use that repository instead of the latest posted patch? No point in testing old code ( I used nested_hstore_and_jsonb-2.patch ). Could you send a link to where that repository is? ( btw, your query is not quite the same as the one I used: select 'x' || ('a=1':: hstore) but your query also crashes my server here so I suppose it triggers the same bug ) thanks, Erik Rijkers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
Thank you, Erik ! Oleg On Mon, Jan 13, 2014 at 12:25 PM, Erik Rijkers e...@xs4all.nl wrote: On Mon, January 13, 2014 00:24, Erik Rijkers wrote: On Sat, January 11, 2014 22:47, Andrew Dunstan wrote: On 01/11/2014 03:03 PM, Erik Rijkers wrote: On Sat, January 11, 2014 20:30, Peter Eisentraut wrote: The documentation doesn't build. corrective patch is here: http://www.postgresql.org/message-id/37b9f104d5a838eec9b75f3668517aa5.squir...@webmail.xs4all.nl It will be in the next version of the patch posted. Attached is another handful of doc-fixes... There are errors in the example expressions in Table F-6. hstore Operators. Attached is a cumulative doc-patch (which includes the changes I sent earlier) which fixes these. I also attach an test perl program that shows the (small) differences in output between what's in that doc table and what one actually gets. (I found these too insignificant to change but perhaps you have a different opinion.) thanks, Erik Rijkers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
Andrew Dunstan wrote: On 01/13/2014 03:25 AM, Erik Rijkers wrote: There are errors in the example expressions in Table F-6. hstore Operators. Attached is a cumulative doc-patch (which includes the changes I sent earlier) which fixes these. I also attach an test perl program that shows the (small) differences in output between what's in that doc table and what one actually gets. (I found these too insignificant to change but perhaps you have a different opinion.) A new version of the patch is attached. It includes all of Erik's docs fixes and a small fix by Alexander Korotkov for hstore hash ops. Interestingly, this also include transaction_commit event triggers. There are also a few PANIC elogs, probably not what's intended. (I was just giving this a quick skim to see if there's support to build JSON objects incrementally from C source, i.e. not have to call functions using the fmgr interface. Apparently that's not the case, but if I'm wrong please let me know.) -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
Andrew, did you run perl script ? Actually, I found, that operator table needs to be fixed. Oleg On Mon, Jan 13, 2014 at 7:36 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/13/2014 03:25 AM, Erik Rijkers wrote: There are errors in the example expressions in Table F-6. hstore Operators. Attached is a cumulative doc-patch (which includes the changes I sent earlier) which fixes these. I also attach an test perl program that shows the (small) differences in output between what's in that doc table and what one actually gets. (I found these too insignificant to change but perhaps you have a different opinion.) A new version of the patch is attached. It includes all of Erik's docs fixes and a small fix by Alexander Korotkov for hstore hash ops. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 01/13/2014 11:03 AM, Alvaro Herrera wrote: Andrew Dunstan wrote: On 01/13/2014 03:25 AM, Erik Rijkers wrote: There are errors in the example expressions in Table F-6. hstore Operators. Attached is a cumulative doc-patch (which includes the changes I sent earlier) which fixes these. I also attach an test perl program that shows the (small) differences in output between what's in that doc table and what one actually gets. (I found these too insignificant to change but perhaps you have a different opinion.) A new version of the patch is attached. It includes all of Erik's docs fixes and a small fix by Alexander Korotkov for hstore hash ops. Interestingly, this also include transaction_commit event triggers. Oh, wow, really? git really did something horrible, or I did inadvertently. This is what comes from using the same directory for multiple development lines :-( Will fix There are also a few PANIC elogs, probably not what's intended. Oleg, Teodor, please address. (I was just giving this a quick skim to see if there's support to build JSON objects incrementally from C source, i.e. not have to call functions using the fmgr interface. Apparently that's not the case, but if I'm wrong please let me know.) Erm, maybe you need the other json patch: http://www.postgresql.org/message-id/52c76b33.1050...@dunslane.net If we need to adjust some of that a bit to make it more friendly for internal use I'm happy to try to do that. Unfortunately, I don't think that's terribly easy for VARIADIC any functions like these. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 01/13/2014 11:16 AM, Oleg Bartunov wrote: Andrew, did you run perl script ? Actually, I found, that operator table needs to be fixed. No. My build machine doesn't actually have DBD::Pg installed. Can you send me a patch if you don't want to push it yourself, or maybe Erik can send a pacth top adjust the table. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Mon, January 13, 2014 18:30, Andrew Dunstan wrote: On 01/13/2014 11:16 AM, Oleg Bartunov wrote: Andrew, did you run perl script ? Actually, I found, that operator table needs to be fixed. No. My build machine doesn't actually have DBD::Pg installed. Can you send me a patch if you don't want to push it yourself, or maybe Erik can send a pacth top adjust the table. [ nested_hstore_and_jsonb-2.patch ] ( centos 6.5, gcc 4.8.2. ) The patch applies compiles with warnings (see below). The opr_sanity test fails during make check: regression.diffs attached. Also attached are changes to hstore.sgml, to operator + functions table, plus some typos. Thanks, Erik Rijkers make jsonfuncs.c: In function each_object_field_end_jsonb: jsonfuncs.c:1328:7: warning: assignment from incompatible pointer type [enabled by default] val = DatumGetPointer(DirectFunctionCall1(jsonb_in, CStringGetDatum(cstr))); ^ jsonfuncs.c: In function elements_array_element_end_jsonb: jsonfuncs.c:1530:8: warning: assignment from incompatible pointer type [enabled by default] jbval = DatumGetPointer(DirectFunctionCall1(jsonb_in, CStringGetDatum(cstr))); ^ make contrib: hstore_io.c: In function array_to_hstore: hstore_io.c:1694:30: warning: result may be used uninitialized in this function [-Wmaybe-uninitialized] PG_RETURN_POINTER(hstoreDump(result)); regression.diffs Description: Binary data --- doc/src/sgml/hstore.sgml.orig 2014-01-14 00:06:30.070883763 +0100 +++ doc/src/sgml/hstore.sgml 2014-01-14 00:58:53.069334810 +0100 @@ -350,7 +350,7 @@ entrytypetext[]//entry entryget values for keys (literalNULL/ if not present)/entry entryliteral'a=gt;x, b=gt;y, c=gt;z'::hstore -gt; ARRAY['c','a']/literal/entry - entryliteral{z,x}/literal/entry + entryliteral{z,x}/literal/entry /row row @@ -422,7 +422,7 @@ entrytypehstore//entry entrydelete key from left operand/entry entryliteral'a=gt;1, b=gt;2, c=gt;3'::hstore - 'b'::text/literal/entry - entryliterala=gt;1, c=gt;3/literal/entry + entryliterala=gt;1, c=gt;3/literal/entry /row row @@ -438,7 +438,7 @@ entrytypehstore//entry entrydelete keys from left operand/entry entryliteral'a=gt;1, b=gt;2, c=gt;3'::hstore - ARRAY['a','b']/literal/entry - entryliteralc=gt;3/literal/entry + entryliteralc=gt;3/literal/entry /row row @@ -446,14 +446,14 @@ entrytypehstore//entry entrydelete matching pairs from left operand/entry entryliteral'a=gt;1, b=gt;2, c=gt;3'::hstore - 'a=gt;4, b=gt;2'::hstore/literal/entry - entryliterala=gt;1, c=gt;3/literal/entry + entryliterala=gt;1, c=gt;3/literal/entry /row row entrytypehstore/ literal#-/ typetext[]//entry entrytypehstore//entry entrydelete key path from left operand/entry - entryliteral'{a =gt; {b =gt; { c =gt; [1,2]}}}'::hstore #- '[a,b,c,0]'/literal/entry + entryliteral'{a =gt; {b =gt; { c =gt; [1,2]}}}'::hstore #- '{a,b,c,0}'/literal/entry entryliterala=gt;{b=gt;{c=gt;[2]}}/literal/entry /row @@ -525,7 +525,7 @@ entrytypehstore/type/entry entryconstruct an typehstore/ from a record or row/entry entryliteralhstore(ROW(1,2))/literal/entry - entryliteralf1=gt;1,f2=gt;2/literal/entry + entryliteralf1=gt;1,f2=gt;2/literal/entry /row row @@ -534,7 +534,7 @@ entryconstruct an typehstore/ from an array, which may be either a key/value array, or a two-dimensional array/entry entryliteralhstore(ARRAY['a','1','b','2']) || hstore(ARRAY[['c','3'],['d','4']])/literal/entry - entryliterala=gt;1, b=gt;2, c=gt;3, d=gt;4/literal/entry + entryliterala=gt;1, b=gt;2, c=gt;3, d=gt;4/literal/entry /row row @@ -707,7 +707,7 @@ entrytypehstore/type/entry entryextract a subset of an typehstore//entry entryliteralslice('a=gt;1,b=gt;2,c=gt;3'::hstore, ARRAY['b','c','x'])/literal/entry - entryliteralb=gt;2, c=gt;3/literal/entry + entryliteralb=gt;2, c=gt;3/literal/entry /row row @@ -766,15 +766,15 @@ entryfunctionreplace(hstore,text[],hstore)/functionindextermprimaryreplace/primary/indexterm/entry entrytypehstore/type/entry entryreplace value at the specified path/entry - entryliteralreplace('a=gt;1,b=gt;{c=gt;3,d=gt;[4,5,6]}'::hstore,'[b,d]', '1')/literal/entry - entryliterala=gt;1, b=gt;{c=gt;3, d=gt;}/literal/entry + entryliteralreplace('a=gt;1,b=gt;{c=gt;3,d=gt;[4,5,6]}'::hstore,'{b,d}', '1')/literal/entry + entryliterala=gt;1, b=gt;{c=gt;3, d=gt;1}/literal/entry /row row entryfunctionconcat_path(hstore,text[],hstore)/functionindextermprimaryconcat_path/primary/indexterm/entry entrytypehstore/type/entry entryconcatenate typehstore/ value at the specified path/entry -
Re: [HACKERS] nested hstore patch
On Sat, January 11, 2014 22:47, Andrew Dunstan wrote: On 01/11/2014 03:03 PM, Erik Rijkers wrote: On Sat, January 11, 2014 20:30, Peter Eisentraut wrote: The documentation doesn't build. corrective patch is here: http://www.postgresql.org/message-id/37b9f104d5a838eec9b75f3668517aa5.squir...@webmail.xs4all.nl It will be in the next version of the patch posted. Attached is another handful of doc-fixes... --- doc/src/sgml/hstore.sgml.orig 2014-01-12 15:37:59.292863864 +0100 +++ doc/src/sgml/hstore.sgml 2014-01-13 00:17:51.454592023 +0100 @@ -55,11 +55,11 @@ The text representation of an typehstore/, used for input and output, may be formatted as scalar values, hash-like values, array-like values, and nested array and hash values. Scalar values are simply strings, numeric - values, booleans, or literalNULL/. Strings continaining whitespace, + values, booleans, or literalNULL/. Strings containing whitespace, commas, literal=/s or literalgt;/s must be double-quoted. To include a double quote or a backslash in a key or value, escape it with a - backslash. Boolean values may be represnted as literaltrue/, literalt/, - literalfalse/, or literalf/. Use quotation marks to represent thes + backslash. Boolean values may be represented as literaltrue/, literalt/, + literalfalse/, or literalf/. Use quotation marks to represent these values as strings. The literalNULL/ keyword is case-insensitive. Double-quote the literalNULL/ to treat it as the ordinary string quoteNULL/quote. Some examples: @@ -87,9 +87,9 @@ /para para - Hashes includes zero or more + Hashes include zero or more replaceablekey/ literal=gt;/ replaceablevalue/ pairs separated - by commas, optionally brackted by curly braces. Keys must be strings and + by commas, optionally bracketed by curly braces. Keys must be strings and may not be literalNULL/; values may be any typehstore/ type, including literalNULL/. Examples: @@ -291,7 +291,7 @@ row entrytypehstore/ literal?gt;/ typeinteger//entry - entrytypebolean//entry + entrytypeboolean//entry entryget boolean value for array index (literalNULL/ if not boolean or not present)/entry entryliteral'[false,null,44]'::hstore ?gt; 0/literal/entry entryliteralf/literal/entry @@ -316,7 +316,7 @@ row entrytypehstore/ literal#?gt;/ typetext[]//entry entrytypeboolean//entry - entryget boolean value for key path (literalNULL/ if not booelan or not present)/entry + entryget boolean value for key path (literalNULL/ if not boolean or not present)/entry entryliteral'foo =gt; {bar =gt; true}'::hstore #?gt; '[foo,bar]'/literal/entry entryliteralt/literal/entry /row @@ -905,7 +905,7 @@ row entryliteralpretty_print//entry - entryAdds add newlines between values and indents nested hashes and arrays./entry + entryAdds newlines between values and indents nested hashes and arrays./entry entryliteralhstore_print('a=gt;t, t=gt;f, arr=gt;[1,2,3]', pretty_print := true)/literal/entry entry programlisting @@ -938,7 +938,7 @@ row entryliteraljson//entry - entryRetuns the value as a JSON string/entry + entryReturns the value as a JSON string/entry entryliteralhstore_print('arr=gt;[1,2,3]', json := true)/literal/entry entryliteralarr: [1, 2, 3]/literal/entry /row @@ -1007,7 +1007,7 @@ para But literalpopulate_record()/ supports more complicated records and nested - typehstore/a values, as well. It makes an effort to convert + typehstore/ values, as well. It makes an effort to convert from typehstore/ data types to PostgreSQL types, including arrays, typejson/, and typehstore/ values: programlisting -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
The documentation doesn't build. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Sat, January 11, 2014 20:30, Peter Eisentraut wrote: The documentation doesn't build. corrective patch is here: http://www.postgresql.org/message-id/37b9f104d5a838eec9b75f3668517aa5.squir...@webmail.xs4all.nl -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 01/11/2014 03:03 PM, Erik Rijkers wrote: On Sat, January 11, 2014 20:30, Peter Eisentraut wrote: The documentation doesn't build. corrective patch is here: http://www.postgresql.org/message-id/37b9f104d5a838eec9b75f3668517aa5.squir...@webmail.xs4all.nl It's been committed at https://github.com/feodor/postgres/commit/a21a4be55a5b12c4bd89b6ab2f77cf32e319de31. It will be in the next version of the patch posted. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Jan 11, 2014, at 1:47 PM, Andrew Dunstan and...@dunslane.net wrote: It's been committed at https://github.com/feodor/postgres/commit/a21a4be55a5b12c4bd89b6ab2f77cf32e319de31. It will be in the next version of the patch posted. Bah! Sorry about that. Habit from decades of typing HTML. David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Thu, Jan 9, 2014 at 5:08 PM, Andrew Dunstan and...@dunslane.net wrote: * I have replicated all the json processing functions for jsonb (although not the json generating functions, such as to_json). Most of these currently work by turning the jsonb back into json and then processing as before. I am sorting out some technical issues and hope to have all of these rewritten to use the native jsonb API in a few days time. * We still need to document jsonb. That too I hope will be done quite shortly. * The jsonb regression test currently contains U+ABCD - I guess we'd better use some hex encoding or whatever for that - unlike json, the jsonb de-serializer dissolves unicode escapes. How does that work if the server encoding isn't UTF-8? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 01/10/2014 01:29 PM, Robert Haas wrote: On Thu, Jan 9, 2014 at 5:08 PM, Andrew Dunstan and...@dunslane.net wrote: * The jsonb regression test currently contains U+ABCD - I guess we'd better use some hex encoding or whatever for that - unlike json, the jsonb de-serializer dissolves unicode escapes. How does that work if the server encoding isn't UTF-8? There is a jsonb_1.out file for the non-utf8 case, just as there is a json_1.out for the same case. Unicode escapes for non-ascii characters are forbidden in jsonb as they are in json, if the encoding isn't utf8. FYI, we are actually using the json lexing and parsing mechanism, so that these types will accept exactly the same inputs. However, since we're not storing json text in jsonb, but instead the decomposed elements, the unicode escapes are resolved in the stored values. I already have a fix for the point above (see https://github.com/feodor/postgres/commit/7d5b8f12747b4a75e8b32914340d07617f1af302) and it will be included in the next version of the patch. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 01/08/2014 04:29 PM, Oleg Bartunov wrote: Attached is a new version of patch, which addresses most issues raised by Andres. It's long holidays in Russia now and it happened that Teodor is traveling with family, so Teodor asked me to reply. Comments in code will be added asap. Oleg, Please merge in the jsonb work and resubmit. See https://github.com/feodor/postgres/commits/jsonb_and_hstore I not that this repo does not apparently contain any of your latest changes. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 01/09/2014 06:12 AM, Andrew Dunstan wrote: Oleg, Please merge in the jsonb work and resubmit. See https://github.com/feodor/postgres/commits/jsonb_and_hstore I note that this repo does not apparently contain any of your latest changes. I'll go further and say that if the Hstore2 patch doesn't support JSONB for 9.4, we should postpone it to 9.5. We really don't want to get into a situation where we need an Hstore3 because we accepted an Hstore2 which needs to be rev'd for JSON. Especially since there's no good reason for the JSON changes not to be merged already. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
I moved patch to the January commitfest (https://commitfest.postgresql.org/action/patch_view?id=1289) . Oleg PS. Kudos to Teodor and his mobile phone, which he used to synchronize branches on github. On Fri, Jan 10, 2014 at 2:08 AM, Andrew Dunstan and...@dunslane.net wrote: On 01/09/2014 02:11 PM, Josh Berkus wrote: On 01/09/2014 06:12 AM, Andrew Dunstan wrote: Oleg, Please merge in the jsonb work and resubmit. See https://github.com/feodor/postgres/commits/jsonb_and_hstore I note that this repo does not apparently contain any of your latest changes. I'll go further and say that if the Hstore2 patch doesn't support JSONB for 9.4, we should postpone it to 9.5. We really don't want to get into a situation where we need an Hstore3 because we accepted an Hstore2 which needs to be rev'd for JSON. Especially since there's no good reason for the JSON changes not to be merged already. After some work by Oleg, for which I'm grateful, and a little more by me, here is a combined patch for the jsonb and nested hstore work. Outstanding issues with the jsonb stuff: * I have replicated all the json processing functions for jsonb (although not the json generating functions, such as to_json). Most of these currently work by turning the jsonb back into json and then processing as before. I am sorting out some technical issues and hope to have all of these rewritten to use the native jsonb API in a few days time. * We still need to document jsonb. That too I hope will be done quite shortly. * The jsonb regression test currently contains U+ABCD - I guess we'd better use some hex encoding or whatever for that - unlike json, the jsonb de-serializer dissolves unicode escapes. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch (sgml typo)
On Wed, January 8, 2014 22:29, Oleg Bartunov wrote: Attached is a new version of patch, which addresses most issues raised by Andres. [ nested_hstore-0.42.patch.gz ] Building documentation fails: openjade:hstore.sgml:1010:18:E: end tag for element A which is not open openjade:hstore.sgml:1011:13:E: document type does not allow element TYPE here openjade:hstore.sgml:1012:8:E: document type does not allow element TYPE here openjade:hstore.sgml:1012:27:E: document type does not allow element TYPE here openjade:hstore.sgml:1013:15:E: document type does not allow element PROGRAMLISTING here openjade:hstore.sgml:1024:8:E: end tag for TYPE omitted, but OMITTAG NO was specified openjade:hstore.sgml:1010:3: start tag was here make: *** [HTML.index] Error 1 make: *** Deleting file `HTML.index' This is caused by a small tag typo. The attached fixes that hstore.sgml typo. thanks, Erikjan --- doc/src/sgml/hstore.sgml.orig 2014-01-08 23:32:29.493548857 +0100 +++ doc/src/sgml/hstore.sgml 2014-01-08 23:33:02.554527949 +0100 @@ -1007,7 +1007,7 @@ para But literalpopulate_record()/ supports more complicated records and nested - typehstore/a values, as well. It makes an effort to convert + typehstore/ values, as well. It makes an effort to convert from typehstore/ data types to PostgreSQL types, including arrays, typejson/, and typehstore/ values: programlisting -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 12/23/13, 9:47 AM, Pavel Stehule wrote: Has anybody looked into how hard it would be to add method notation to postgreSQL, so that instead of calling getString(hstorevalue, n) we could use hstorevalue.getString(n) yes, I played with it some years ago. I ended early, there was a problem with parser - when I tried append a new rule. And because there was not simple solution, I didn't continue. But it can be nice feature - minimally for plpgsql coders. Isn't there also some major problem with differentiating between schema/table/field with that too? I recall discussion along those lines, though maybe it was for the idea of recursive schemas. -- Jim C. Nasby, Data Architect j...@nasby.net 512.569.9461 (cell) http://jim.nasby.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Fri, Dec 20, 2013 at 6:16 PM, David E. Wheeler da...@justatheory.com wrote: * New operators: + `hstore - int`: Get string value at array index (starting at 0) + `hstore ^ text`:Get numeric value for key + `hstore ^ int`: Get numeric value at array index + `hstore ? text`:Get boolean value for key + `hstore ? int`: Get boolean value at array index + `hstore # text[]`: Get string value for key path + `hstore #^ text[]`: Get numeric value for key path + `hstore #? text[]`: Get boolean value for key path + `hstore % text`:Get hstore value for key + `hstore % int`: Get hstore value at array index + `hstore #% text[]`: Get hstore value for key path + `hstore ? int`: Does hstore contain array index + `hstore #? text[]`: Does hstore contain key path + `hstore - int`: Delete index from left operand + `hstore #- text[]`: Delete key path from left operand Although in some ways there's a certain elegance to this, it also sorta looks like punctuation soup. I can't help wondering whether we'd be better off sticking to function names. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 12/23/2013 12:28 PM, Robert Haas wrote: On Fri, Dec 20, 2013 at 6:16 PM, David E. Wheeler da...@justatheory.com wrote: * New operators: + `hstore - int`: Get string value at array index (starting at 0) + `hstore ^ text`:Get numeric value for key + `hstore ^ int`: Get numeric value at array index + `hstore ? text`:Get boolean value for key + `hstore ? int`: Get boolean value at array index + `hstore # text[]`: Get string value for key path + `hstore #^ text[]`: Get numeric value for key path + `hstore #? text[]`: Get boolean value for key path + `hstore % text`:Get hstore value for key + `hstore % int`: Get hstore value at array index + `hstore #% text[]`: Get hstore value for key path + `hstore ? int`: Does hstore contain array index + `hstore #? text[]`: Does hstore contain key path + `hstore - int`: Delete index from left operand + `hstore #- text[]`: Delete key path from left operand Although in some ways there's a certain elegance to this, it also sorta looks like punctuation soup. I can't help wondering whether we'd be better off sticking to function names. Has anybody looked into how hard it would be to add method notation to postgreSQL, so that instead of calling getString(hstorevalue, n) we could use hstorevalue.getString(n) -- Hannu Krosing PostgreSQL Consultant Performance, Scalability and High Availability 2ndQuadrant Nordic OÜ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
Hello 2013/12/23 Hannu Krosing ha...@2ndquadrant.com On 12/23/2013 12:28 PM, Robert Haas wrote: On Fri, Dec 20, 2013 at 6:16 PM, David E. Wheeler da...@justatheory.com wrote: * New operators: + `hstore - int`: Get string value at array index (starting at 0) + `hstore ^ text`:Get numeric value for key + `hstore ^ int`: Get numeric value at array index + `hstore ? text`:Get boolean value for key + `hstore ? int`: Get boolean value at array index + `hstore # text[]`: Get string value for key path + `hstore #^ text[]`: Get numeric value for key path + `hstore #? text[]`: Get boolean value for key path + `hstore % text`:Get hstore value for key + `hstore % int`: Get hstore value at array index + `hstore #% text[]`: Get hstore value for key path + `hstore ? int`: Does hstore contain array index + `hstore #? text[]`: Does hstore contain key path + `hstore - int`: Delete index from left operand + `hstore #- text[]`: Delete key path from left operand Although in some ways there's a certain elegance to this, it also sorta looks like punctuation soup. I can't help wondering whether we'd be better off sticking to function names. Has anybody looked into how hard it would be to add method notation to postgreSQL, so that instead of calling getString(hstorevalue, n) we could use hstorevalue.getString(n) yes, I played with it some years ago. I ended early, there was a problem with parser - when I tried append a new rule. And because there was not simple solution, I didn't continue. But it can be nice feature - minimally for plpgsql coders. Regards Pavel -- Hannu Krosing PostgreSQL Consultant Performance, Scalability and High Availability 2ndQuadrant Nordic OÜ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Dec 23, 2013, at 6:28 AM, Robert Haas wrote: On Fri, Dec 20, 2013 at 6:16 PM, David E. Wheeler da...@justatheory.com wrote: * New operators: + `hstore - int`: Get string value at array index (starting at 0) + `hstore ^ text`:Get numeric value for key + `hstore ^ int`: Get numeric value at array index + `hstore ? text`:Get boolean value for key + `hstore ? int`: Get boolean value at array index + `hstore # text[]`: Get string value for key path + `hstore #^ text[]`: Get numeric value for key path + `hstore #? text[]`: Get boolean value for key path + `hstore % text`:Get hstore value for key + `hstore % int`: Get hstore value at array index + `hstore #% text[]`: Get hstore value for key path + `hstore ? int`: Does hstore contain array index + `hstore #? text[]`: Does hstore contain key path + `hstore - int`: Delete index from left operand + `hstore #- text[]`: Delete key path from left operand Although in some ways there's a certain elegance to this, it also sorta looks like punctuation soup. I can't help wondering whether we'd be better off sticking to function names. The key thing is making it easy for people to easily chain calls to their nested hstore objects, and I think these operators accomplish that. Some of them are fairly intuitive, and I think as a community if we have a) good docs, b) good blog posts on how to use nested hstore, and c) provides clear instructions @ PG events on how to use it, it would be okay, though some things, i.e. extracting the key by a path, might be better being in a function anyway. However, having it as an operator might encourage more usage, only because people tend to think that functions will slow my query down. My only concern is the consistency with the generally accepted standard of JSON and with the upcoming jsonb type. I'm not sure if the jsonb API has been defined yet, but it would be great to keep consistency between nested hstore and jsonb so people don't have to learn two different access systems. Data extraction from JSON is often done by the dot operator in implementations, and depending on the language you are in, there are ways to add / test existence / remove objects from the JSON blob. Being able to extract objects from nested hstore / JSON using the dot operator would be simple and intuitive and general well-understood, but of course there are challenges with doing that in PG and well, proper SQL. Jonathan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Nov 12, 2013, at 10:35 AM, Teodor Sigaev teo...@sigaev.ru wrote: Hi! Attatched patch adds nesting feature, types (string, boll and numeric values), arrays and scalar to hstore type. My apologies for not getting to this sooner, work has been a bit nutty. The truth is that I reviewed this patch quite a bit a month back, mostly so I could write documentation, the results of which are included in this patch. And I'm super excited for what's to come in the next iteration, as I hear that Teodor and Andrew are hard at work adding jsonb as a binary-compatible JSON data type. Meanwhile, for this version, a quick overview of what has changed since 9.2. Contents Purpose == Improved Data Type Support -- * Added data type support for values. Previously they could only be strings or NULL, but with this patch they can also be numbers or booleans. * Added array support. Values can be arrays of other values. The format for arrays is a bracketed, comma-delimited list. * Added nesting support. hstore values can themselves be hstores. Nested hstores are wrapped in braces, but the root-level hstore is not (for compatibility with the format of previous versions of hstore). * An hstore value is no longer required to be an hstore object. It can now be any scalar value. These three items make the basic format feature-complete with JSON. Here's an example where the values are scalars: =% SELECT 'foo'::hstore, 'hi \bob\'::hstore, '1.0'::hstore, 'true'::hstore, NULL::hstore; hstore |hstore| hstore | hstore | hstore +--+++ foo | hi \bob\ | 1.0| t | And here are a couple of arrays with strings, numbers, booleans, and NULLs: SELECT '[k,v]'::hstore, '[1.0, hi there, false, null]'::hstore; hstore | hstore + [k, v] | [1.0, hi there, f, NULL] Here's a complicated example formatted with `hstore.pretty_print` enabled. =% SET hstore.pretty_print=true; =% SELECT '{ type = Feature, bbox = [-180.0, -90.0, 180.0, 90.0], geometry = { type = Polygon, coordinates = [[ [-180.0, 10.0], [20.0, 90.0], [180.0, -5.0], [-30.0, -90.0] ]] } }'::hstore; hstore -- bbox=+ [ + -180.0, + -90.0, + 180.0, + 90.0+ ], + type=Feature, + geometry=+ { + type=Polygon, + coordinates= + [ + [ + [ + -180.0, + 10.0+ ], + [ + 20.0, + 90.0+ ], + [ + 180.0, + -5.0+ ], + [ + -30.0, + -90.0 + ] + ] + ] + } So, exact feature parity with the JSON data type. * hstore.pretty_print is a new GUC, specifically to allow an HSTORE value to be pretty-printed. There is also a function to pretty-print, so we might be able to just do away with the GUC. Interface - * New operators: + `hstore - int`: Get string value at array index (starting at 0) + `hstore ^ text`:Get numeric value for key + `hstore ^ int`: Get numeric value at array index + `hstore ? text`:Get boolean value for key + `hstore ? int`: Get boolean value at array index + `hstore # text[]`: Get string value for key path + `hstore #^ text[]`: Get numeric value for key path + `hstore #? text[]`: Get boolean value for key path + `hstore % text`:Get hstore value for key + `hstore % int`: Get hstore value at array index + `hstore #% text[]`: Get hstore value for key path + `hstore ? int`: Does hstore contain array index + `hstore #? text[]`: Does hstore contain key path + `hstore - int`: Delete index from left operand + `hstore #- text[]`: Delete key path from left operand * New functions: + `hstore(text)`: Make a text scalar hstore + `hstore(numeric)`: Make a numeric scalar hstore + `hstore(boolean)`: Make a boolean scalar hstore + `hstore(text, hstore)`: Make a nested hstore + `hstore(text, numeric)`:Make an hstore with a
Re: [HACKERS] nested hstore patch
On 2013-12-20 15:16:30 -0800, David E. Wheeler wrote: But for the hstore feature itself, I think the current interface and features are ready to go. I think this patch needs significant amount of work because it can be considered ready for committer. I found the list of issues in http://archives.postgresql.org/message-id/20131118163633.GE20305%40awork2.anarazel.de within 10 minutes, indicating that it clearly cannot be ready yet. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 11/12/13, 1:35 PM, Teodor Sigaev wrote: Hi! Attatched patch adds nesting feature, types (string, boll and numeric values), arrays and scalar to hstore type. Documentation doesn't build: openjade:hstore.sgml:206:16:E: document type does not allow element VARLISTENTRY here; assuming missing VARIABLELIST start-tag Compiler warnings: hstore_io.c: In function 'array_to_hstore': hstore_io.c:1736:29: error: 'result' may be used uninitialized in this function [-Werror=maybe-uninitialized] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Nov 20, 2013, at 6:19 AM, Peter Eisentraut pete...@gmx.net wrote: openjade:hstore.sgml:206:16:E: document type does not allow element VARLISTENTRY here; assuming missing VARIABLELIST start-tag Thanks, I fixed this one. David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Thu, Nov 14, 2013 at 05:50:02PM +0100, Hannu Krosing wrote: If we could somehow turn old json into a text domain with json syntax check (which it really is up to 9.3) via pg_upgrade that would be great. It would be the required for pg_dump to have some swicth to output different typename in CREATE TABLE and similar. I don't think pg_upgrade isn't in a position to handle this. I think it would require a script to be run after pg_upgrade completes. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Wed, Nov 13, 2013 at 6:59 PM, Hannu Krosing ha...@2ndquadrant.com wrote: I remember strong voices in support of *not* normalising json, so that things like {a:1,a:true, a:b, a:none} would go through the system unaltered, for claimed standard usage of json as processing instructions. That is as source code which can possibly converted to JavaScript Object and not something that would come out of serialising of any existing JavaScript Object. Yeah, as the guy who wrote the original version of the JSON type, which works just exactly like the XML type does, I stronly object to changing the behavior. And doubly so now that it's released, as we would be breaking backward compatibility. I suggest we add another type, maybe jsobj, which has input and output as standard JSON but which is defined from the start to be equivalent of existing object and not preservable source code to such object. I think this was the consensus solution when this was last discussed, and I support it. There is similar space for a binary XML data type if someone feels like implementing it. I think the names that were proposed previously were something like jsonb and xmlb. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Tue, Nov 19, 2013 at 10:51:06AM -0500, Robert Haas wrote: I think this was the consensus solution when this was last discussed, and I support it. There is similar space for a binary XML data type if someone feels like implementing it. I think the names that were proposed previously were something like jsonb and xmlb. The natural name is OBJSON, meaning object JSON, because as PostgreSQL people, we have to double-use letters wherever possible. ;-) -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 11/19/2013 10:51 AM, Robert Haas wrote: I suggest we add another type, maybe jsobj, which has input and output as standard JSON but which is defined from the start to be equivalent of existing object and not preservable source code to such object. I think this was the consensus solution when this was last discussed, and I support it. There is similar space for a binary XML data type if someone feels like implementing it. I think the names that were proposed previously were something like jsonb and xmlb. I think that's the consensus position on a strategy. JSONB seems to be the current winner min the name stakes. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Tue, Nov 19, 2013 at 10:55 AM, Bruce Momjian br...@momjian.us wrote: On Tue, Nov 19, 2013 at 10:51:06AM -0500, Robert Haas wrote: I think this was the consensus solution when this was last discussed, and I support it. There is similar space for a binary XML data type if someone feels like implementing it. I think the names that were proposed previously were something like jsonb and xmlb. The natural name is OBJSON, meaning object JSON, because as PostgreSQL people, we have to double-use letters wherever possible. ;-) Personally, I think the patch author should just run ps auxww | md5 | sed 's/^[^0-9]//' | cut -c1-8 and call the new data type by the resulting name. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 11/19/2013 11:00 AM, Robert Haas wrote: On Tue, Nov 19, 2013 at 10:55 AM, Bruce Momjian br...@momjian.us wrote: On Tue, Nov 19, 2013 at 10:51:06AM -0500, Robert Haas wrote: I think this was the consensus solution when this was last discussed, and I support it. There is similar space for a binary XML data type if someone feels like implementing it. I think the names that were proposed previously were something like jsonb and xmlb. The natural name is OBJSON, meaning object JSON, because as PostgreSQL people, we have to double-use letters wherever possible. ;-) Personally, I think the patch author should just run ps auxww | md5 | sed 's/^[^0-9]//' | cut -c1-8 and call the new data type by the resulting name. My personal vote goes for marmaduke. I've been dying to call some builtin object that for ages. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
Hi, On 2013-11-12 22:35:31 +0400, Teodor Sigaev wrote: Attatched patch adds nesting feature, types (string, boll and numeric values), arrays and scalar to hstore type. I took a quick peek at this: * You cannot simply catch and ignore errors by doing + PG_TRY(); + { + n = DatumGetNumeric(DirectFunctionCall3(numeric_in, CStringGetDatum(s-val), 0, -1)); + } + PG_CATCH(); + { + n = NULL; + } + PG_END_TRY(); That skips cleanup and might ignore some errors (think memory allocation failures). But why do you even want to silently ignore errors there? * Shouldn't the checks for v-size be done before filling the datastructures in makeHStoreValueArray() and makeHStoreValuePairs()? * could you make ORDER_PAIRS() a function instead of a macro? It's pretty long and there's no reason not to use a function. * You call numeric_recv via recvHStoreValue via recvHStore without checks on the input length. That seems - without having checked it in detail - a good way to read unrelated memory. Generally ISTM the input needs to be more carefully checked in the whole recv function. * There's quite some new new, completely uncommented, code. Especially in hstore_op.c. * the _PG_init you added should probably do a EmitWarningsOnPlaceholders(). * why does hstore need it's own atoi? * shouldn't all the function prototypes be marked as externs? * Lots of trailing whitespaces, quite some long lines, cuddly braces, ... * I think hstore_compat.c's header should be updated. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 11/14/2013 01:32 AM, David E. Wheeler wrote: On Nov 13, 2013, at 3:59 PM, Hannu Krosing ha...@2ndquadrant.com wrote: I remember strong voices in support of *not* normalising json, so that things like {a:1,a:true, a:b, a:none} would go through the system unaltered, for claimed standard usage of json as processing instructions. That is as source code which can possibly converted to JavaScript Object and not something that would come out of serialising of any existing JavaScript Object. My recollection from PGCon was that there was consensus to normalize on the way in -- Great news! I remember advocating this approach in the mailing lists but having been out-voted based on current real-world usage out there :) or at least, if we switched to a binary representation as proposed by Oleg Teodor, it was not worth the hassle to try to keep it. Very much agree. For the source code approach I'd recommend text type with maybe a check that it is possible to convert it to json. -- Hannu Krosing PostgreSQL Consultant Performance, Scalability and High Availability 2ndQuadrant Nordic OÜ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 11/14/2013 03:21 AM, Hannu Krosing wrote: On 11/14/2013 01:32 AM, David E. Wheeler wrote: On Nov 13, 2013, at 3:59 PM, Hannu Krosing ha...@2ndquadrant.com wrote: I remember strong voices in support of *not* normalising json, so that things like {a:1,a:true, a:b, a:none} would go through the system unaltered, for claimed standard usage of json as processing instructions. That is as source code which can possibly converted to JavaScript Object and not something that would come out of serialising of any existing JavaScript Object. My recollection from PGCon was that there was consensus to normalize on the way in -- Great news! I remember advocating this approach in the mailing lists but having been out-voted based on current real-world usage out there :) or at least, if we switched to a binary representation as proposed by Oleg Teodor, it was not worth the hassle to try to keep it. Very much agree. For the source code approach I'd recommend text type with maybe a check that it is possible to convert it to json. I don't think you and David are saying the same thing. AIUI he wants one JSON type and is prepared to discard text preservation (duplicate keys and key order). You want two json types, one of which would feature text preservation. Correct me if I'm wrong. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 11/14/2013 01:47 PM, Andrew Dunstan wrote: On 11/14/2013 03:21 AM, Hannu Krosing wrote: On 11/14/2013 01:32 AM, David E. Wheeler wrote: On Nov 13, 2013, at 3:59 PM, Hannu Krosing ha...@2ndquadrant.com wrote: I remember strong voices in support of *not* normalising json, so that things like {a:1,a:true, a:b, a:none} would go through the system unaltered, for claimed standard usage of json as processing instructions. That is as source code which can possibly converted to JavaScript Object and not something that would come out of serialising of any existing JavaScript Object. My recollection from PGCon was that there was consensus to normalize on the way in -- Great news! I remember advocating this approach in the mailing lists but having been out-voted based on current real-world usage out there :) or at least, if we switched to a binary representation as proposed by Oleg Teodor, it was not worth the hassle to try to keep it. Very much agree. For the source code approach I'd recommend text type with maybe a check that it is possible to convert it to json. I don't think you and David are saying the same thing. AIUI he wants one JSON type and is prepared to discard text preservation (duplicate keys and key order). You want two json types, one of which would feature text preservation. I actually *want* the same thing that David wants, but I think that Merlin has valid concerns about backwards compatibility. If we have let this behaviour in, it is not nice to break several uses of it now. If we could somehow turn old json into a text domain with json syntax check (which it really is up to 9.3) via pg_upgrade that would be great. It would be the required for pg_dump to have some swicth to output different typename in CREATE TABLE and similar. Correct me if I'm wrong. cheers andrew -- Hannu Krosing PostgreSQL Consultant Performance, Scalability and High Availability 2ndQuadrant Nordic OÜ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 11/12/13, 1:35 PM, Teodor Sigaev wrote: Attatched patch adds nesting feature, types (string, boll and numeric values), arrays and scalar to hstore type. Could you check your email client for next time? It's sending Content-Type: application/x-tar for a *.patch.gz file. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 11/13/2013 01:37 AM, Andrew Dunstan wrote: On 11/12/2013 01:35 PM, Teodor Sigaev wrote: Hi! Attatched patch adds nesting feature, types (string, boll and numeric values), arrays and scalar to hstore type. All new features are described in PGConf.EU talk http://www.sai.msu.su/~megera/postgres/talks/hstore-dublin-2013.pdf (since PGCon some features was added). Patch includes: 1 implementaion SRF_RETURN_NEXT_NULL() 2 contrib/hstore changes 3 docs of new hstore module (many thanks to David E. Wheeler david.whee...@pgexperts.com) In current state patch is in WIP status, for short period I plan to move support of binary nested structure to core to share binary representation for hstore and json types. Thanks, Teodor. As soon as we have that shared binary representation available, I will be working on adapting it to JSON. As I remember from earlier discussions, current json has some artefacts that some people want to preserve and which are incompatible with hstore approach where you have actual object behind the serialisation. I remember strong voices in support of *not* normalising json, so that things like {a:1,a:true, a:b, a:none} would go through the system unaltered, for claimed standard usage of json as processing instructions. That is as source code which can possibly converted to JavaScript Object and not something that would come out of serialising of any existing JavaScript Object. I suggest we add another type, maybe jsobj, which has input and output as standard JSON but which is defined from the start to be equivalent of existing object and not preservable source code to such object. Cheers -- Hannu Krosing PostgreSQL Consultant Performance, Scalability and High Availability 2ndQuadrant Nordic OÜ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On Nov 13, 2013, at 3:59 PM, Hannu Krosing ha...@2ndquadrant.com wrote: I remember strong voices in support of *not* normalising json, so that things like {a:1,a:true, a:b, a:none} would go through the system unaltered, for claimed standard usage of json as processing instructions. That is as source code which can possibly converted to JavaScript Object and not something that would come out of serialising of any existing JavaScript Object. My recollection from PGCon was that there was consensus to normalize on the way in -- or at least, if we switched to a binary representation as proposed by Oleg Teodor, it was not worth the hassle to try to keep it. I suggest we add another type, maybe jsobj, which has input and output as standard JSON but which is defined from the start to be equivalent of existing object and not preservable source code to such object. -1 Let's try to keep this simple. See also VARCHAR and VARCHAR2 on Oracle. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nested hstore patch
On 11/12/2013 01:35 PM, Teodor Sigaev wrote: Hi! Attatched patch adds nesting feature, types (string, boll and numeric values), arrays and scalar to hstore type. All new features are described in PGConf.EU talk http://www.sai.msu.su/~megera/postgres/talks/hstore-dublin-2013.pdf (since PGCon some features was added). Patch includes: 1 implementaion SRF_RETURN_NEXT_NULL() 2 contrib/hstore changes 3 docs of new hstore module (many thanks to David E. Wheeler david.whee...@pgexperts.com) In current state patch is in WIP status, for short period I plan to move support of binary nested structure to core to share binary representation for hstore and json types. Thanks, Teodor. As soon as we have that shared binary representation available, I will be working on adapting it to JSON. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers