[Sorry - I posted this in CFCommunity already but all the action seems to be
over here...]
I've worked on it some more and have something that, on paper, seems good to
me. I've built a JavaScript Serializer (but haven't yet begun to tackle the
deserializer). I've created and validated the XSD as well.
The basic idea, again, is a solution which can transfer structured data
between JavaScript (or, really, any client - but primarily client-side
JavaScript) and a server. By "structured data" I mean things like arrays
and objects (structs) nested however deeply you want.
My (simplified) goals (and complaints against other solutions) were:
+) Something that's easy to parse for JavaScript. SOAP is NOT easy to
parse (which is, I think, why there's no generalized SOAP parser for JS).
+) Something that maintains general data types. SOAP maintains complex
datatypes (based on Java or C+). JSON doesn't offer data typing at all. I
thought we needed a middle ground.
+) Something in XML. Nearly all modern languages deal with XML natively.
The DOM makes dealing with it in JavaScript easy (if tedious). It seemed
the way to go.
+) Something which could be described using modern XML tools. WDDX, for
example, can't be described using an XSD... which may be stupid on the W3C's
part but is true nonetheless. But XML/XSD provides a basic level of
validation that would have to be built from scratch otherwise.
+) Something that's appropriate for both download AND upload traffic.
+Often
the acts of serializing or deserializing things is left to the server which
outputs raw JavaScript code. Or JavaScript is left to receive structured
data but is left with only flat form fields to respond.
+) Finally, and not the most important, I wanted something that would be
relatively small. JSON is VERY small (adding only 5-10% to most data
packets) while SOAP or WDDX were VERY big (something quintupling the size of
data packets). Again, I thought there should be a middle ground without
losing capability.
I wanted something to take a complex JavaScript object and pass it, intact,
to a server which could take the resulting object, modify it and pass it
back, intact.
I wanted something JavaScript could both serialize and deserialize
relatively quickly and easily. Most importantly I wanted something that _I_
could build and understand in JavaScript and that wouldn't take me a month
to do!
Lastly I wanted something so simple that it would be easy to recreate in
other languages as well. And yet powerful enough not to leave (most) people
accepting compromises in their data.
So, I've come up with this. I call it "dataML" (have you got something
better?):
Three tags only (I decided to minimize them to abbreviations since I'm only
using two tags):
<dataML>: This is the wrapper tag for the packet. It can contain, first,
any number or zero <md> tags and one <d> tag.
<d>: "Data". This tag contains the data items. It can contain as many
other <d> tags as you like in an orgasmic orgy of nesting. (By having only
one tag we eliminate problems that XSD has with randomly appearing child
tags.)
It take three attributes:
type: The type of the data. I settled on using JavaScript's
generalized types (plus the addition of "binary"). If not provided the data
type will default to "string". Possible values are:
+) "object" (equivalent to a CF Struct)
+) "array" (single dimensional with sparse arrays supported)
+) "null"
+) "undefined" (I'm not sure if this is actually needed yet)
+) "string"
+) "number"
+) "boolean" ("true" or "false" are the only acceptable
values.)
+) "date"
+) "binary" (BASE64 Data)
+) "function" (representing a JavaScript function.
fields: For objects this is a comma-delimited list of the properties
of the object (or, for CF, the keys of a struct). For arrays, if used at
all, it would be the indexes filled in a sparse array. It's ignored for all
other data types.
label: A label which corresponds to a label using in a <md>
("metadata") tag. If used the other attributes will be pulled from the
metadata.
<md>: "MetaData". This allows us to create a set of attributes to be
applied later to multiple data items. While its use may slow down
serialization/deserialization using it well can shrink the size of the
resulting packet tremendously.
We can use <md> to "pre-populate" attribute values for <d> tags.
It takes one attribute:
label: A label which identifies the metadata. This will be used in
the "label" attribute of <d> tags. <md> tags must appear BEFORE <d> tags.
But examples probably say it better.
A simple, two property object could be displayed like this:
<dataML>
<d type="object" fields="fname, mname, lname, ">
<d type="string">Jim</d>
<d type="null" />
<d type="string">Davis</d>
</d>
</dataML>
Where a simple record set could look like this:
<dataML>
<d type="array">
<d type="object" fields="text, value,">
<d type="string">Fund One</d>
<d type="string">Fund01</d>
</d>
<d type="object" fields="text, value,">
<d type="string">Fund Two</d>
<d type="string">Fund02</d>
</d>
<d type="object" fields="text, value,">
<d type="string">Fund Three</d>
<d type="string">Fund03</d>
</d>
</d>
</dataML>
But over large numbers of records the repetition of the metadata would
inflate the packet terribly. So we use the <md> tag like so:
<dataML>
<md label="option">
<d type="object" fields="text, value,">
<d type="string" />
<d type="string" />
</d>
</md>
<d type="array">
<d label="option" >
<d>Fund One</d>
<d>Fund01</d>
</d>
<d label="option" >
<d>Fund Two</d>
<d>Fund02</d>
</d>
<d label="option" >
<d>Fund Three</d>
<d>Fund03</d>
</d>
</d>
</dataML>
This shrinks the serialized version of the data tremendously when you want
to return large numbers of similar objects (as with a record set).
I'm still not completely settled on the rules for <md>. It does complicate
things up but I like the idea so much I'm trying to work through them.
Essentially I see a two pass-parse of this: the first would add the
metadata-defined attributes to the labeled data items, the second would
parse the packet normally.
But there are questions. Should "default data" be allowed in meta data (if
you fill in the content of the tags should that content be used if no
content is provided in the <d> tags)?
Should metadata for the contents of an array automatically apply to ALL
children of that array? In other words should this be legal:
<dataML>
<md label="options">
<d type="array">
<d type="object" fields="text, value,">
<d type="string" />
<d type="string" />
</d>
</d>
</md>
<d label="options">
<d>
<d>Fund One</d>
<d>Fund01</d>
</d>
<d>
<d>Fund Two</d>
<d>Fund02</d>
</d>
<d>
<d>Fund Three</d>
<d>Fund03</d>
</d>
</d>
</dataML>
Should such a thing force all children of the array to be the same?
Or should the <md> tag apply ONLY to objects?
Anyway...
In most cases you won't need the <md> at all. It's ONLY needed to save
space. All deserializers should understand it, but serializers don't have
to implement it.
(In any case I doubt it would be easy to implement the <md> tag in a general
serializer anyway: if you want to use it I think it would mean custom code
for specific purposes...)
So - any comments?
Any data structures you think this can't handle? Any aspects you think are
unimplementable or just plain stupid?
Come on! Tear it to shreds! ;^)
Once it's done, of course, all code (I'll be doing at least JavaScript and
ColdFusion versions) will be made public under the BSD license.
Thanks in advance.
Jim Davis
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Find out how CFTicket can increase your company's customer support
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49
Message: http://www.houseoffusion.com/lists.cfm/link=i:4:215549
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
