I've worked on it some more and have something that, on paper, seems good to me. I've built a basic JavaScript Serializer (but haven't yet begun to tackle the deserializer). I've created and validated the XSD as well.
The basic idea, again, is a solution which can transfer structured data between JavaScript (or, really, any client - but primarily client-side JavaScript) and a server. By "structured data" I mean things like arrays and objects (structs) nested however deeply you want. My (simplified) goals (and complaints against other solutions) were: +) Something that's easy to parse for JavaScript. SOAP is NOT easy to parse (which is, I think, why there's no generalized SOAP parser for JS). +) Something that maintains general data types. SOAP maintains complex datatypes (based on Java or C+). JSON doesn't offer data typing at all. I thought we needed a middle ground. +) Something in XML. Nearly all modern languages deal with XML natively. The DOM makes dealing with it in JavaScript easy (if tedious). It seemed the way to go. +) Something which could be described using modern XML tools. WDDX, for example, can't be described using an XSD... which may be stupid on the W3C's part but is true nonetheless. But XML/XSD provides a basic level of validation that would have to be built from scratch otherwise. +) Something that's appropriate for both download AND upload traffic. Often the acts of serializing or deserializing things is left to the server which outputs raw JavaScript code. Or JavaScript is left to receive structured data but is left with only flat form fields to respond. +) Finally, and not the most important, I wanted something that would be relatively small. JSON is VERY small (adding only 5-10% to most data packets) while SOAP or WDDX were VERY big (something quintupling the size of data packets). Again, I thought there should be a middle ground without losing capability. I wanted something to take a complex JavaScript object and pass it, intact, to a server which could take the resulting object, modify it and pass it back, intact. I wanted something JavaScript could both serialize and deserialize relatively quickly and easily. Most importantly I wanted something that _I_ could build and understand in JavaScript and that wouldn't take me a month to do! Lastly I wanted something so simple that it would be easy to recreate in other languages as well. And yet powerful enough not to leave (most) people accepting compromises in their data. So, I've come up with this. I call it "dataML" (have you got something better?): Three tags only (I decided to minimize them to abbreviations since I'm only using two tags): <dataML>: This is the wrapper tag for the packet. It can contain, first, any number or zero <md> tags and one <d> tag. <d>: "Data". This tag contains the data items. It can contain as many other <d> tags as you like in an orgasmic orgy of nesting. (By having only one tag we eliminate problems that XSD has with randomly appearing child tags.) It take three attributes: type: The type of the data. I settled on using JavaScript's generalized types (plus the addition of "binary"). If not provided the data type will default to "string". Possible values are: +) "object" (equivalent to a CF Struct) +) "array" (single dimensional with sparse arrays supported) +) "null" +) "undefined" (I'm not sure if this is actually needed yet) +) "string" +) "number" +) "boolean" ("true" or "false" are the only acceptable values.) +) "date" +) "binary" (BASE64 Data) +) "function" (representing a JavaScript function. fields: For objects this is a comma-delimited list of the properties of the object (or, for CF, the keys of a struct). For arrays, if used at all, it would be the indexes filled in a sparse array. It's ignored for all other data types. label: A label which corresponds to a label using in a <md> ("metadata") tag. If used the other attributes will be pulled from the metadata. <md>: "MetaData". This allows us to create a set of attributes to be applied later to multiple data items. While its use may slow down serialization/deserialization using it well can shrink the size of the resulting packet tremendously. We can use <md> to "pre-populate" attribute values for <d> tags. It takes one attribute: label: A label which identifies the metadata. This will be used in the "label" attribute of <d> tags. <md> tags must appear BEFORE <d> tags. But examples probably say it better. A simple, two property object could be displayed like this: <dataML> <d type="object" fields="fname, mname, lname, "> <d type="string">Jim</d> <d type="null" /> <d type="string">Davis</d> </d> </dataML> Where a simple record set could look like this: <dataML> <d type="array"> <d type="object" fields="text, value,"> <d type="string">Fund One</d> <d type="string">Fund01</d> </d> <d type="object" fields="text, value,"> <d type="string">Fund Two</d> <d type="string">Fund02</d> </d> <d type="object" fields="text, value,"> <d type="string">Fund Three</d> <d type="string">Fund03</d> </d> </d> </dataML> But over large numbers of records the repetition of the metadata would inflate the packet terribly. So we use the <md> tag like so: <dataML> <md label="option"> <d type="object" fields="text, value,"> <d type="string" /> <d type="string" /> </d> </md> <d type="array"> <d label="option" > <d>Fund One</d> <d>Fund01</d> </d> <d label="option" > <d>Fund Two</d> <d>Fund02</d> </d> <d label="option" > <d>Fund Three</d> <d>Fund03</d> </d> </d> </dataML> This shrinks the serialized version of the data tremendously when you want to return large numbers of similar objects (as with a record set). I'm still not completely settled on the rules for <md>. It does complicate things up but I like the idea so much I'm trying to work through them. Essentially I see a two pass-parse of this: the first would add the metadata-defined attributes to the labeled data items, the second would parse the packet normally. But there are questions. Should "default data" be allowed in meta data (if you fill in the content of the tags should that content be used if no content is provided in the <d> tags)? Should metadata for the contents of an array automatically apply to ALL children of that array? In other words should this be legal: <dataML> <md label="options"> <d type="array"> <d type="object" fields="text, value,"> <d type="string" /> <d type="string" /> </d> </d> </md> <d label="options"> <d> <d>Fund One</d> <d>Fund01</d> </d> <d> <d>Fund Two</d> <d>Fund02</d> </d> <d> <d>Fund Three</d> <d>Fund03</d> </d> </d> </dataML> Should such a thing force all children of the array to be the same? Or should the <md> tag apply ONLY to objects? Anyway... In most cases you won't need the <md> at all. It's ONLY needed to save space. All deserializers should understand it, but serializers don't have to implement it. (In any case I doubt it would be easy to implement the <md> tag in a general serializer anyway: if you want to use it I think it would mean custom code for specific purposes...) So - any comments? Any data structures you think this can't handle? Any aspects you think are unimplementable or just plain stupid? Come on! Tear it to shreds! ;^) Once it's done, of course, all code (I'll be doing at least JavaScript and ColdFusion versions) will be made public under the BSD license. Thanks in advance. Jim Davis ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Discover CFTicket - The leading ColdFusion Help Desk and Trouble Ticket application http://www.houseoffusion.com/banners/view.cfm?bannerid=48 Message: http://www.houseoffusion.com/lists.cfm/link=i:5:170188 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/5 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:5 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.5 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54