Re: [HACKERS] PostgreSQL and SOAP, suggestions?
[EMAIL PROTECTED] kirjutas N, 03.04.2003 kell 02:01: mlw wrote: I think you are interpreting the spec a bit too restrictively. The syntax is fairly rigid, but the spec has a great degree of flexibility. I agree that, syntactically, it must work through a parser, but there is lots of room to be flexible. This is /exactly/ the standard problem with SOAP. There is enough flexibility that there are differing approaches associated, generally speaking, with IBM versus Microsoft whereby it's easy to generate SOAP requests that work fine with one that break with the other. Do you know of some: a) standard conformance tests b) recommended best practices for being compatible with all mainstream implementations (I'd guess a good approach would be to generate very strictly conformant code but accept all that you can, even if against pedantic reading of the spec) - Hannu ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
Hannu Krosing wrote: [EMAIL PROTECTED] kirjutas N, 03.04.2003 kell 02:01: mlw wrote: I think you are interpreting the spec a bit too restrictively. The syntax is fairly rigid, but the spec has a great degree of flexibility. I agree that, syntactically, it must work through a parser, but there is lots of room to be flexible. This is /exactly/ the standard problem with SOAP. There is enough flexibility that there are differing approaches associated, generally speaking, with IBM versus Microsoft whereby it's easy to generate SOAP requests that work fine with one that break with the other. Do you know of some: a) standard conformance tests Off the top of my head, no, but I bet it is a goole away. If you know any good links, I'd love to know. I have been working off the W3C spec. b) recommended best practices for being compatible with all mainstream implementations (I'd guess a good approach would be to generate very strictly conformant code but accept all that you can, even if against pedantic reading of the spec) I have been planning to test the whole thing with a few .NET applications. I am currently using expat to parse the output to ensure that it all works correcty. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
[EMAIL PROTECTED] kirjutas N, 03.04.2003 kell 02:01: mlw wrote: I think you are interpreting the spec a bit too restrictively. The syntax is fairly rigid, but the spec has a great degree of flexibility. I agree that, syntactically, it must work through a parser, but there is lots of room to be flexible. This is /exactly/ the standard problem with SOAP. There is enough flexibility that there are differing approaches associated, generally speaking, with IBM versus Microsoft whereby it's easy to generate SOAP requests that work fine with one that break with the other. Do you know of some: a) standard conformance tests b) recommended best practices for being compatible with all mainstream implementations (I'd guess a good approach would be to generate very strictly conformant code but accept all that you can, even if against pedantic reading of the spec) The problem with a) is that SOAP, unlike CORBA, doesn't have the notion of standardized language bindings. That makes it tough to be sure that your implementation is standard in any meaningful way in the first place. The best practices have involved scripting up interoperability tests where they construct sets of functions with varying data types and verify that my client implementation can talk to your server implementation, and vice-versa. And when you run into problems, you chip off bits of code until the block of stone starts looking like an elephant. In order to have confidence of interoperability, you have to test your client library against all the servers you care about, or vice-versa. That's definitely not the same thing as being a conformance test. Trying to be really strict doesn't seem to be a viable strategy, as far as I can see... -- (concatenate 'string cbbrowne @ntlug.org) http://www3.sympatico.ca/cbbrowne/wp.html The cost of living has just gone up another dollar a quart. -- W.C. Fields ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
I have been planning to test the whole thing with a few .NET applications. I am currently using expat to parse the output to ensure that it all works correcty. That, unfortunately, probably implies that your implementation is almost totally non-interoperable. You should put out of your mind the notion of being correct. Being correct is pretty irrelevant if 80% of the requests that come from a VB.NET client fail because Microsoft implemented part of their request differently than what you interpreted as correct. The point is that correctness isn't the thing you need to aim for; what you should aim for is interoperability with the important client implementations. SOAP::Lite, .NET, probably some Java ones, C++ ones, and such. Nobody does correctness testing; they do interoperability tests where they try to submit requests to Apache AXIS, .NET, WebSphere, and the lot of other important implementations. If you're testing a server (as is the case here), then the point is to run tests with a bunch of clients. Head to the SOAP::Lite and Axis projects; you'll see matrices describing this sort of thing... -- (reverse (concatenate 'string ac.notelrac.teneerf@ 454aa)) http://www.ntlug.org/~cbbrowne/advocacy.html Fear leads to anger. Anger leads to hate. Hate leads to using Windows NT for mission-critical applications. --- What Yoda *meant* to say ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
On Thu, Apr 03, 2003 at 07:54:13AM -0500, [EMAIL PROTECTED] wrote: I have been planning to test the whole thing with a few .NET applications. I am currently using expat to parse the output to ensure that it all works correcty. That, unfortunately, probably implies that your implementation is almost totally non-interoperable. You should put out of your mind the notion of being correct. Being correct is pretty irrelevant if 80% of the requests that come from a VB.NET client fail because Microsoft implemented part of their request differently than what you interpreted as correct. The point is that correctness isn't the thing you need to aim for; what you should aim for is interoperability with the important client implementations. SOAP::Lite, .NET, probably some Java ones, C++ ones, and such. Nobody does correctness testing; they do interoperability tests where they try to submit requests to Apache AXIS, .NET, WebSphere, and the lot of other important implementations. If you're testing a server (as is the case here), then the point is to run tests with a bunch of clients. Head to the SOAP::Lite and Axis projects; you'll see matrices describing this sort of thing... Hmmm. Can I reiterate my support of XML-RPC here? g -Jay 'Eraserhead' Felice -- (reverse (concatenate 'string ac.notelrac.teneerf@ 454aa)) http://www.ntlug.org/~cbbrowne/advocacy.html Fear leads to anger. Anger leads to hate. Hate leads to using Windows NT for mission-critical applications. --- What Yoda *meant* to say ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
mlw kirjutas T, 01.04.2003 kell 15:29: Hannu Krosing wrote: [EMAIL PROTECTED] kirjutas E, 31.03.2003 kell 19:52: Actually, as far as I am aware, the header is for metadata, i.e. it is the place to describe the data being returned. Did you read the SOAP spec ? yes ??? What you have come up with _is_not_ a SOAP v1.1 message at all. It does use some elements with similar names but from different namespace. the SOAP Envelope, Header and Body elemants must be from namespace http://schemas.xmlsoap.org/soap/envelope/ Per section 3 paragraph 2 of SOAP spec a conforming SOAP processor MUST discard a message that has incorrect namespace. ?xml version = 1.0? mwssql:Envelope xmlns:mwssql=http://www.mohawksoft.com/mwssql/envelope; mwssql:Header The SOAP-ENV:Header is a generic mechanism for adding features to a SOAP message in a decentralized manner without prior agreement between the communicating parties. SOAP defines a few attributes that can be used to indicate who should deal with a feature and whether it is optional or mandatory (see section 4.2). The SOAP-ENV:Body is a container for mandatory information intended for the ultimate recipient of the message (see section 4.3). SOAP defines one element for the body, which is the Fault element used for reporting errors. The Header element is encoded as the first immediate child element of the SOAP Envelope XML element. All immediate child elements of the Header element are called header entries. The encoding rules for header entries are as follows: 1. A header entry is identified by its fully qualified element name, which consists of the namespace URI and the local name. All immediate child elements of the SOAP Header element MUST be namespace-qualified. ... An example is a header with an element identifier of Transaction, a mustUnderstand value of 1, and a value of 5. This would be encoded as follows: SOAP-ENV:Header t:Transaction xmlns:t=some-URI SOAP-ENV:mustUnderstand=1 5 /t:Transaction /SOAP-ENV:Header exec:sqlupdate cgrpairs set ratio=0 where srcitem=100098670/exec:sql exec:affected2657/exec:affected qry:sqlselect * from ztitles limit 2/qry:sql qry:ROWSET qry:ROW columns=28 where are namespaces exec:, qry: abd t: defined ? Hannu ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
Hannu Krosing wrote: mlw kirjutas T, 01.04.2003 kell 15:29: Hannu Krosing wrote: [EMAIL PROTECTED] kirjutas E, 31.03.2003 kell 19:52: Actually, as far as I am aware, the header is for metadata, i.e. it is the place to describe the data being returned. Did you read the SOAP spec ? yes ??? What you have come up with _is_not_ a SOAP v1.1 message at all. It does use some elements with similar names but from different namespace. the SOAP Envelope, Header and Body elemants must be from namespace http://schemas.xmlsoap.org/soap/envelope/ [snip] Hmm, I read SHOULD and MAY in the spec, assuming that it was not MUST are you saying it is invalid if I do not use the SOAP URIs for the name spaces? If so, no big deal, I'll change them. As for defining the namespaces, yea that's easy enough, just tack on an attribute. I still don't see where putting the field definitions in the soap header is an invalid use of that space. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
Andrew Dunstan writes: If the intention is to use field names as (local) tag names, how will you handle the case where the field name isn't a valid XML name? Of course, one could do some sort of mapping (replace illegal chars with _, for example) but then you can't be 100% certain that you haven't generated a collision, I should think. The SQL/XML draft specifies an reversible escape mechanism. Basically, when mapping an SQL identifier to an XML name you replace problematic characters with an escape sequence based on the Unicode code point, like _x2A3B_. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
mlw writes: That function looks great, but what happens if you need to return 1 million records? The same thing that happens with any set-returning function: memory exhaustion. I have an actual libpq program which performs a query against a server, and will stream out the XML, so the number of records has very little affect on efficiency. I think the table2xml function is great for 99% of all the queries, but for those huge resultsets, I think it may be problematic. What do you think? Clearly, my approach is not sufficient if you need to handle big result sets. But perhaps a compromise based on cursors could be designed so that large parts of the format can be managed centrally. Such as: DECLARE foo CURSOR FOR SELECT ... ; -- gives you the XML Schema for the result set SELECT xmlschema_from_cursor(foo); -- gives you ones row (row.../row) SELECT xmldata_from_cursor(foo); -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
mlw kirjutas K, 02.04.2003 kell 15:56: Hannu Krosing wrote: What you have come up with _is_not_ a SOAP v1.1 message at all. It does use some elements with similar names but from different namespace. the SOAP Envelope, Header and Body elemants must be from namespace http://schemas.xmlsoap.org/soap/envelope/ [snip] Hmm, I read SHOULD and MAY in the spec, assuming that it was not MUST are you saying it is invalid if I do not use the SOAP URIs for the name spaces? If so, no big deal, I'll change them. AFAICS you can _leave_out_ the namespace, but not put in another, nonconforming namespace. As for defining the namespaces, yea that's easy enough, just tack on an attribute. I still don't see where putting the field definitions in the soap header is an invalid use of that space. It is not strictly nonconforming, just not the intended use of transparently adding new info: 4.2 SOAP Header SOAP provides a flexible mechanism for extending a message in a decentralized and modular way without prior knowledge between the communicating parties. Typical examples of extensions that can be implemented as header entries are authentication, transaction management, payment etc. I.e. the intended use of *SOAP* Header is *not* defining the structure of the message but is rather something similar to e-mail (rfc822) Headers. The XML way of defining a message is using a DTD, XML-schema, Relax NG schema or somesuch, either embedded (forbidden for DTD's in SOAP) or referenced. Also for me the following: The Header element is encoded as the first immediate child element of the SOAP Envelope XML element. All immediate child elements of the Header element are called header entries. The encoding rules for header entries are as follows: 1. A header entry is identified by its fully qualified element name, which consists of the namespace URI and the local name. All immediate child elements of the SOAP Header element MUST be namespace-qualified. describes an element with a full embedded URI, not just namespace-qualified tagname, but I may be reading it wrong and the namespace could be defined at outer level. But defining namespace at the outer level is counterintuitive for cases where the header element is to be processed and removed by some SOAP intermediary. Also this seems to support *not* using Header for essensial structure definitions: 4.3.1 Relationship between SOAP Header and Body While the Header and Body are defined as independent elements, they are in fact related. The relationship between a body entry and a header entry is as follows: A body entry is semantically equivalent to a header entry intended for the default actor and with a SOAP mustUnderstand attribute with a value of 1. The default actor is indicated by not using the actor attribute (see section 4.2.2). This suggests that putting the structure definition as 1-st Body element and data as second would be equivalent to putting structure in Header - Hannu ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
Hannu Krosing wrote: mlw kirjutas K, 02.04.2003 kell 15:56: Hannu Krosing wrote: What you have come up with _is_not_ a SOAP v1.1 message at all. It does use some elements with similar names but from different namespace. the SOAP Envelope, Header and Body elemants must be from namespace http://schemas.xmlsoap.org/soap/envelope/ [snip] Hmm, I read SHOULD and MAY in the spec, assuming that it was not MUST are you saying it is invalid if I do not use the SOAP URIs for the name spaces? If so, no big deal, I'll change them. AFAICS you can _leave_out_ the namespace, but not put in another, nonconforming namespace. [snip] I think you are interpreting the spec a bit too restrictively. The syntax is fairly rigid, but the spec has a great degree of flexibility. I agree that, syntactically, it must work through a parser, but there is lots of room to be flexible. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
mlw wrote: I think you are interpreting the spec a bit too restrictively. The syntax is fairly rigid, but the spec has a great degree of flexibility. I agree that, syntactically, it must work through a parser, but there is lots of room to be flexible. This is /exactly/ the standard problem with SOAP. There is enough flexibility that there are differing approaches associated, generally speaking, with IBM versus Microsoft whereby it's easy to generate SOAP requests that work fine with one that break with the other. For a pretty simple example of a longstanding bug that has never been fixed, see: http://sourceforge.net/tracker/index.php?func=detailaid=559324group_id=26590atid=387667 The precis: The SOAP implementation used by the XMethods folks to publish stock prices is buggy, rejecting perfectly legitimate messages submitted using ZSI (a Python SOAP implementation). The bug isn't with ZSI; it is quite clearly with the server, apparently implemented in Java using one of the EJB frameworks. In practice, what happens is that since that service is fairly popular, particularly for sample applications, the implementors of SOAP libraries wind up coding around the bugs. The problem is that it gets difficult to tell the difference between bugs and variations in interpretations of the standards. If the specs were more strictly defined, it would be a lot easier to use SOAP, because you wouldn't be left puzzling over whether the interoperability problems you're having are: a) Problems with the client; b) Problems with the server; c) Problems with interpretation of specs; d) ... The vast degree to which messages can get rewritten behind your back adds to the fun. Of course, it's only fun if you *enjoy* having interoperability problems... -- If this was helpful, http://svcs.affero.net/rm.php?r=cbbrowne rate me http://www.ntlug.org/~cbbrowne/soap.html He who laughs last thinks slowest. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
Hannu Krosing wrote: [EMAIL PROTECTED] kirjutas E, 31.03.2003 kell 19:52: Actually, as far as I am aware, the header is for metadata, i.e. it is the place to describe the data being returned. Did you read the SOAP spec ? yes The description of the fields isn't the actual data retrieved, so it doesn't belong in the body, so it should go into the header. That is logical, but this is not what the spec tells. This is exactly what the spec calles for. The spec, at least 1.1, says very little about what should not be in the header. For an XML request, it should carry. It is very particular about soap header attributes, but header contents is very flexable. Also the spec requires immediate child elements of SOAP:Header to have full namespace URI's. Yup, that was a bug. And another question - why do you have the namespace MWSSQL defined but never used ? That was part of the same bug as above, it now outputs this: ?xml version = 1.0? mwssql:Envelope xmlns:mwssql=http://www.mohawksoft.com/mwssql/envelope; mwssql:Header exec:sqlupdate cgrpairs set ratio=0 where srcitem=100098670/exec:sql exec:affected2657/exec:affected qry:sqlselect * from ztitles limit 2/qry:sql qry:ROWSET qry:ROW columns=28 t:acdundefined/t:acd t:muzenbrundefined/t:muzenbr t:cat2undefined/t:cat2 t:cat3undefined/t:cat3 t:cat4undefined/t:cat4 t:performerundefined/t:performer t:performer2undefined/t:performer2 t:titleundefined/t:title t:artist1undefined/t:artist1 t:engineerundefined/t:engineer t:producerundefined/t:producer t:labelnameundefined/t:labelname t:catalogundefined/t:catalog t:distributundefined/t:distribut t:releasedundefined/t:released t:origrelundefined/t:origrel t:nbrdiscsundefined/t:nbrdiscs t:sparundefined/t:spar t:minutesundefined/t:minutes t:secondsundefined/t:seconds t:monostereoundefined/t:monostereo t:studioliveundefined/t:studiolive t:availableundefined/t:available t:previewsundefined/t:previews t:pnotesundefined/t:pnotes t:artistidundefined/t:artistid t:datasrcundefined/t:datasrc t:extidundefined/t:extid /qry:ROW /qry:ROWSET /mwssql:Header mwssql:Body ROWSET columns=28 rows=2 ROW ROWID=0 acdP/acd muzenbr68291/muzenbr cat2Performer/cat2 cat3Jazz Instrument/cat3 cat4Guitar/cat4 performerSteve Khan/performer performer2Khan, Steve/performer2 titleEvidence/title artist1/artist1 engineer/engineer producer/producer labelnameNovus/labelname catalog3074/catalog distributBMG/distribut released02/13/1990/released origreln/a/origrel nbrdiscs1/nbrdiscs sparn/a/spar minutes/minutes seconds/seconds monostereoStereo/monostereo studioliveStudio/studiolive availableN/available previews/previews pnotes/pnotes artistid100025343/artistid datasrc1/datasrc extid68291/extid /ROW ROW ROWID=1 acdP/acd muzenbr67655/muzenbr cat2Collection/cat2 cat3Jazz Instrument/cat3 cat4/cat4 performerVarious Artists/performer performer2Various Artists/performer2 titleMetropolitan Opera House Jam Session/title artist1/artist1 engineer/engineer producer/producer labelnameJazz Anthology/labelname catalog550212/catalog distributn/a/distribut released1992/released origreln/a/origrel nbrdiscs1/nbrdiscs sparn/a/spar minutes/minutes seconds/seconds monostereoMono/monostereo studioliveLive/studiolive availableN/available previews/previews pnotes/pnotes artistid100050450/artistid datasrc1/datasrc extid67655/extid /ROW /ROWSET /mwssql:Body /mwssql:Envelope ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
Out of curiousity, what is the purpose of putting the qry:ROWSET description into the message at all (header or not)? Isn't it a perfectly valid SOAP message (and just as parseable) with that removed? I freely admit to not being a soap expert, but similar SOAP messages I generate from queries seem to work fine without this metadata. Is having it required by some part of the SOAP spec I don't understand? Thanks! On Tue, 2003-04-01 at 05:29, mlw wrote: That was part of the same bug as above, it now outputs this: ?xml version = 1.0? mwssql:Envelope xmlns:mwssql=http://www.mohawksoft.com/mwssql/envelope; mwssql:Header exec:sqlupdate cgrpairs set ratio=0 where srcitem=100098670/exec:sql exec:affected2657/exec:affected qry:sqlselect * from ztitles limit 2/qry:sql qry:ROWSET qry:ROW columns=28 t:acdundefined/t:acd t:muzenbrundefined/t:muzenbr t:cat2undefined/t:cat2 t:cat3undefined/t:cat3 t:cat4undefined/t:cat4 t:performerundefined/t:performer t:performer2undefined/t:performer2 t:titleundefined/t:title t:artist1undefined/t:artist1 t:engineerundefined/t:engineer t:producerundefined/t:producer t:labelnameundefined/t:labelname t:catalogundefined/t:catalog t:distributundefined/t:distribut t:releasedundefined/t:released t:origrelundefined/t:origrel t:nbrdiscsundefined/t:nbrdiscs t:sparundefined/t:spar t:minutesundefined/t:minutes t:secondsundefined/t:seconds t:monostereoundefined/t:monostereo t:studioliveundefined/t:studiolive t:availableundefined/t:available t:previewsundefined/t:previews t:pnotesundefined/t:pnotes t:artistidundefined/t:artistid t:datasrcundefined/t:datasrc t:extidundefined/t:extid /qry:ROW /qry:ROWSET /mwssql:Header mwssql:Body ROWSET columns=28 rows=2 ROW ROWID=0 acdP/acd muzenbr68291/muzenbr cat2Performer/cat2 cat3Jazz Instrument/cat3 cat4Guitar/cat4 performerSteve Khan/performer performer2Khan, Steve/performer2 titleEvidence/title artist1/artist1 engineer/engineer producer/producer labelnameNovus/labelname catalog3074/catalog distributBMG/distribut released02/13/1990/released origreln/a/origrel nbrdiscs1/nbrdiscs sparn/a/spar minutes/minutes seconds/seconds monostereoStereo/monostereo studioliveStudio/studiolive availableN/available previews/previews pnotes/pnotes artistid100025343/artistid datasrc1/datasrc extid68291/extid /ROW ROW ROWID=1 acdP/acd muzenbr67655/muzenbr cat2Collection/cat2 cat3Jazz Instrument/cat3 cat4/cat4 performerVarious Artists/performer performer2Various Artists/performer2 titleMetropolitan Opera House Jam Session/title artist1/artist1 engineer/engineer producer/producer labelnameJazz Anthology/labelname catalog550212/catalog distributn/a/distribut released1992/released origreln/a/origrel nbrdiscs1/nbrdiscs sparn/a/spar minutes/minutes seconds/seconds monostereoMono/monostereo studioliveLive/studiolive availableN/available previews/previews pnotes/pnotes artistid100050450/artistid datasrc1/datasrc extid67655/extid /ROW /ROWSET /mwssql:Body /mwssql:Envelope -- Steve Wampler [EMAIL PROTECTED] National Solar Observatory ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
I can certainly imagine cases for processing where having the field names and other metadata up front (maybe add type info, nullable, etc instead of just undefined) would be useful. here's another question: If the intention is to use field names as (local) tag names, how will you handle the case where the field name isn't a valid XML name? Of course, one could do some sort of mapping (replace illegal chars with _, for example) but then you can't be 100% certain that you haven't generated a collision, I should think. andrew - Original Message - From: Steve Wampler [EMAIL PROTECTED] To: mlw [EMAIL PROTECTED] Cc: Hannu Krosing [EMAIL PROTECTED]; [EMAIL PROTECTED]; Postgres-hackers [EMAIL PROTECTED] Sent: Tuesday, April 01, 2003 12:40 PM Subject: Re: [HACKERS] PostgreSQL and SOAP, suggestions? Out of curiousity, what is the purpose of putting the qry:ROWSET description into the message at all (header or not)? Isn't it a perfectly valid SOAP message (and just as parseable) with that removed? I freely admit to not being a soap expert, but similar SOAP messages I generate from queries seem to work fine without this metadata. Is having it required by some part of the SOAP spec I don't understand? Thanks! On Tue, 2003-04-01 at 05:29, mlw wrote: That was part of the same bug as above, it now outputs this: ?xml version = 1.0? mwssql:Envelope xmlns:mwssql=http://www.mohawksoft.com/mwssql/envelope; mwssql:Header exec:sqlupdate cgrpairs set ratio=0 where srcitem=100098670/exec:sql exec:affected2657/exec:affected qry:sqlselect * from ztitles limit 2/qry:sql qry:ROWSET qry:ROW columns=28 t:acdundefined/t:acd t:muzenbrundefined/t:muzenbr t:cat2undefined/t:cat2 t:cat3undefined/t:cat3 t:cat4undefined/t:cat4 t:performerundefined/t:performer t:performer2undefined/t:performer2 t:titleundefined/t:title t:artist1undefined/t:artist1 t:engineerundefined/t:engineer t:producerundefined/t:producer t:labelnameundefined/t:labelname t:catalogundefined/t:catalog t:distributundefined/t:distribut t:releasedundefined/t:released t:origrelundefined/t:origrel t:nbrdiscsundefined/t:nbrdiscs t:sparundefined/t:spar t:minutesundefined/t:minutes t:secondsundefined/t:seconds t:monostereoundefined/t:monostereo t:studioliveundefined/t:studiolive t:availableundefined/t:available t:previewsundefined/t:previews t:pnotesundefined/t:pnotes t:artistidundefined/t:artistid t:datasrcundefined/t:datasrc t:extidundefined/t:extid /qry:ROW /qry:ROWSET /mwssql:Header mwssql:Body ROWSET columns=28 rows=2 ROW ROWID=0 acdP/acd muzenbr68291/muzenbr cat2Performer/cat2 cat3Jazz Instrument/cat3 cat4Guitar/cat4 performerSteve Khan/performer performer2Khan, Steve/performer2 titleEvidence/title artist1/artist1 engineer/engineer producer/producer labelnameNovus/labelname catalog3074/catalog distributBMG/distribut released02/13/1990/released origreln/a/origrel nbrdiscs1/nbrdiscs sparn/a/spar minutes/minutes seconds/seconds monostereoStereo/monostereo studioliveStudio/studiolive availableN/available previews/previews pnotes/pnotes artistid100025343/artistid datasrc1/datasrc extid68291/extid /ROW ROW ROWID=1 acdP/acd muzenbr67655/muzenbr cat2Collection/cat2 cat3Jazz Instrument/cat3 cat4/cat4 performerVarious Artists/performer performer2Various Artists/performer2 titleMetropolitan Opera House Jam Session/title artist1/artist1 engineer/engineer producer/producer labelnameJazz Anthology/labelname catalog550212/catalog distributn/a/distribut released1992/released origreln/a/origrel nbrdiscs1/nbrdiscs sparn/a/spar minutes/minutes seconds/seconds monostereoMono/monostereo studioliveLive/studiolive availableN/available previews/previews pnotes/pnotes artistid100050450/artistid datasrc1/datasrc extid67655/extid /ROW /ROWSET /mwssql:Body /mwssql:Envelope -- ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
I can certainly imagine cases for processing where having the field names and other metadata up front (maybe add type info, nullable, etc instead of just undefined) would be useful. here's another question: If the intention is to use field names as (local) tag names, how will you handle the case where the field name isn't a valid XML name? Of course, one could do some sort of mapping (replace illegal chars with _, for example) but then you can't be 100% certain that you haven't generated a collision, I should think. I'm not sure, I have to really research how to handle that case. I have been simply doing a %hex translation on characters that do not conform to XML, that may actually be good enough(tm). As for the field names being undefined, if you can find a way to get the field types without having to specify a binary cursor I'd like that. Admitedly, I have not looked very hard. This is a small part of a bigger project. The SQL/XML provider currently supports PG and ODBC. The web services project, which contains the SQL/XML provider, has a bunch of other services. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
mlw writes:
Given a HTTP formatted query:
GET http://localhost:8181/pgmuze?query=select+*+from+zsong+limit+2;
The output is entered below.
That looks a lot like the SQL/XML-style output plus a SOAP header. Below
is the output that I get from the SQL/XML function that I wrote. A simple
XSLT stylesheet should do the trick for you.
Btw., I also have an XSLT stylesheet that can make an HTML table out of
this output and I have a table function that can generate a virtual table
from this output.
= select table2xml('select * from products');
?xml version='1.0'?
table
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xsi:noNamespaceSchemaLocation='#' !-- XXX this needs to be fixed --
xsd:schema
xmlns:xsd='http://www.w3.org/2001/XMLSchema'
xmlns:sqlxml='http://www.iso-standards.org/mra/9075/2001/12/sqlxml'
xsd:import
namespace='http://www.iso-standards.org/mra/9075/2001/12/sqlxml'
schemaLocation='http://www.iso-standards.org/mra/9075/2001/12/sqlxml.xsd' /
xsd:simpleType name='peter.pg_catalog.text'
xsd:restriction base='xsd:string'
xsd:maxLength value='MLIT' / !-- XXX needs actual value --
/xsd:restriction
/xsd:simpleType
xsd:simpleType name='INTEGER'
xsd:restriction base='xsd:integer'
xsd:maxInclusive value='2147483647'/
xsd:minInclusive value='-2147483648'/
/xsd:restriction
/xsd:simpleType
xsd:simpleType name='NUMERIC'
xsd:restriction base='xsd:decimal'
xsd:totalDigits value='PLIT'/ !-- XXX needs actual values --
xsd:fractionDigits value='SLIT'/
/xsd:restriction
/xsd:simpleType
xsd:complexType name='RowType'
xsd:sequence
xsd:element name='name' type='peter.pg_catalog.text'
nillable='true'/xsd:element
xsd:element name='category' type='INTEGER' nillable='true'/xsd:element
xsd:element name='price' type='NUMERIC' nillable='true'/xsd:element
/xsd:sequence
/xsd:complexType
xsd:complexType name='TableType'
xsd:sequence
xsd:element name='row' type='RowType' minOccurs='0' maxOccurs='unbounded' /
/xsd:sequence
/xsd:complexType
xsd:element name='table' type='TableType' /
/xsd:schema
row
namescrewdriver/name
category3/category
price7.99/price
/row
row
namedrill/name
category9/category
price12.49/price
/row
/table
--
Peter Eisentraut [EMAIL PROTECTED]
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
That function looks great, but what happens if you need to return 1
million records? Wouldn't you exhaust all the memory in the server? Or
can you stream it somehow?
I have an actual libpq program which performs a query against a server,
and will stream out the XML, so the number of records has very little
affect on efficiency. I think the table2xml function is great for 99% of
all the queries, but for those huge resultsets, I think it may be
problematic.
What do you think?
BTW, I routinely have queries that return millions of rows.
Peter Eisentraut wrote:
mlw writes:
Given a HTTP formatted query:
GET http://localhost:8181/pgmuze?query=select+*+from+zsong+limit+2;
The output is entered below.
That looks a lot like the SQL/XML-style output plus a SOAP header. Below
is the output that I get from the SQL/XML function that I wrote. A simple
XSLT stylesheet should do the trick for you.
Btw., I also have an XSLT stylesheet that can make an HTML table out of
this output and I have a table function that can generate a virtual table
from this output.
= select table2xml('select * from products');
?xml version='1.0'?
table
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xsi:noNamespaceSchemaLocation='#' !-- XXX this needs to be fixed --
xsd:schema
xmlns:xsd='http://www.w3.org/2001/XMLSchema'
xmlns:sqlxml='http://www.iso-standards.org/mra/9075/2001/12/sqlxml'
xsd:import
namespace='http://www.iso-standards.org/mra/9075/2001/12/sqlxml'
schemaLocation='http://www.iso-standards.org/mra/9075/2001/12/sqlxml.xsd' /
xsd:simpleType name='peter.pg_catalog.text'
xsd:restriction base='xsd:string'
xsd:maxLength value='MLIT' / !-- XXX needs actual value --
/xsd:restriction
/xsd:simpleType
xsd:simpleType name='INTEGER'
xsd:restriction base='xsd:integer'
xsd:maxInclusive value='2147483647'/
xsd:minInclusive value='-2147483648'/
/xsd:restriction
/xsd:simpleType
xsd:simpleType name='NUMERIC'
xsd:restriction base='xsd:decimal'
xsd:totalDigits value='PLIT'/ !-- XXX needs actual values --
xsd:fractionDigits value='SLIT'/
/xsd:restriction
/xsd:simpleType
xsd:complexType name='RowType'
xsd:sequence
xsd:element name='name' type='peter.pg_catalog.text' nillable='true'/xsd:element
xsd:element name='category' type='INTEGER' nillable='true'/xsd:element
xsd:element name='price' type='NUMERIC' nillable='true'/xsd:element
/xsd:sequence
/xsd:complexType
xsd:complexType name='TableType'
xsd:sequence
xsd:element name='row' type='RowType' minOccurs='0' maxOccurs='unbounded' /
/xsd:sequence
/xsd:complexType
xsd:element name='table' type='TableType' /
/xsd:schema
row
namescrewdriver/name
category3/category
price7.99/price
/row
row
namedrill/name
category9/category
price12.49/price
/row
/table
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
mlw kirjutas E, 31.03.2003 kell 03:43: Given a HTTP formatted query: GET http://localhost:8181/pgmuze?query=select+*+from+zsong+limit+2; The output is entered below. Questions: Is there a way, without spcifying a binary cursor, to get the data types associated with columns? Right now I am just using undefined, as the ODBC version works. Anyone see any basic improvements needed? ?xml version = 1.0? soap:Envelope xmlns:MWSSQL=http://www.mohawksoft.com/MWSSQL/envelope; soap:Header !-- Fields in set -- Columns count=9 The SOAP 1.1 spec specifies (p4.2) the following about SOAP Header: The encoding rules for header entries are as follows: 1. A header entry is identified by its fully qualified element name, which consists of the namespace URI and the local name. All immediate child elements of the SOAP Header element MUST be namespace-qualified. I'm not sure that SOAP Header is the right place for Query header info, as the header is meant for: SOAP provides a flexible mechanism for extending a message in a decentralized and modular way without prior knowledge between the communicating parties. Typical examples of extensions that can be implemented as header entries are authentication, transaction management, payment etc. So the definition of structure should probably be inside SOAP:Body . --- Hannu ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
Actually, as far as I am aware, the header is for metadata, i.e. it is the place to describe the data being returned. The description of the fields isn't the actual data retrieved, so it doesn't belong in the body, so it should go into the header. mlw kirjutas E, 31.03.2003 kell 03:43: Given a HTTP formatted query: GET http://localhost:8181/pgmuze?query=select+*+from+zsong+limit+2; The output is entered below. Questions: Is there a way, without spcifying a binary cursor, to get the data types associated with columns? Right now I am just using undefined, as the ODBC version works. Anyone see any basic improvements needed? ?xml version = 1.0? soap:Envelope xmlns:MWSSQL=http://www.mohawksoft.com/MWSSQL/envelope; soap:Header !-- Fields in set -- Columns count=9 The SOAP 1.1 spec specifies (p4.2) the following about SOAP Header: The encoding rules for header entries are as follows: 1. A header entry is identified by its fully qualified element name, which consists of the namespace URI and the local name. All immediate child elements of the SOAP Header element MUST be namespace-qualified. I'm not sure that SOAP Header is the right place for Query header info, as the header is meant for: SOAP provides a flexible mechanism for extending a message in a decentralized and modular way without prior knowledge between the communicating parties. Typical examples of extensions that can be implemented as header entries are authentication, transaction management, payment etc. So the definition of structure should probably be inside SOAP:Body . --- Hannu ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
[EMAIL PROTECTED] kirjutas E, 31.03.2003 kell 19:52: Actually, as far as I am aware, the header is for metadata, i.e. it is the place to describe the data being returned. Did you read the SOAP spec ? The description of the fields isn't the actual data retrieved, so it doesn't belong in the body, so it should go into the header. That is logical, but this is not what the spec tells. Also the spec requires immediate child elements of SOAP:Header to have full namespace URI's. And another question - why do you have the namespace MWSSQL defined but never used ? - Hannu ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PostgreSQL and SOAP, suggestions?
Given a HTTP formatted query: GET http://localhost:8181/pgmuze?query=select+*+from+zsong+limit+2; The output is entered below. Questions: Is there a way, without spcifying a binary cursor, to get the data types associated with columns? Right now I am just using undefined, as the ODBC version works. Anyone see any basic improvements needed? ?xml version = 1.0? soap:Envelope xmlns:MWSSQL=http://www.mohawksoft.com/MWSSQL/envelope; soap:Header !-- Fields in set -- Columns count=9 muzenbrundefined/muzenbr discundefined/disc trkundefined/trk songundefined/song artistidundefined/artistid acdundefined/acd trackidundefined/trackid datasrcundefined/datasrc extidundefined/extid /Columns /soap:Header soap:Body ROWSET columns=9 rows=2 ROW ROWID=0 muzenbr424965/muzenbr disc1/disc trk5/trk songWrite My Name In The Groove/song artistid100021391/artistid acdA/acd trackid203429573/trackid datasrc1/datasrc extid203429573/extid /ROW ROW ROWID=1 muzenbr177516/muzenbr disc1/disc trk1/trk songPapa Was A Rolling Stone/song artistid10411/artistid acdP/acd trackid2/trackid datasrc1/datasrc extid2/extid /ROW /ROWSET /soap:Body /soap:Envelope Steve Wampler wrote: On Fri, 2003-03-28 at 14:39, mlw wrote: I was thinking of using SOAP over HTTP as the protocol, and a minimalist version at best. If the people want more let them add it. I have an HTTP service class in my open source library. It would br trivial to accept a SQL query formatted as a GET request, and then execute the query and, using libpq, format the result as XML. It should be simple enough to do. It would be easy. I've done something similar (using ODBC to get to PostgreSQL) - but using a language none of the rest of you are likely to be interested in (Unicon). Works just fine, though the implementation (deliberately, by personal preference) avoids accepting arbitrary SQL statements from SOAP clients, instead forcing the clients to use an RPC interface so I can do sanity checking in the Unicon [which I know better than I know PostgreSQL...] SOAP servers. I, too, opted for a 'minimal-SOAP' implementation. A 'real' implementation boggles the mind. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
