On 07/15/13 13:51, Chris Angelico wrote: > So the only bit you still need is: How do you transmit this across the > network? Since it's now all just bytes, that's easy enough to do, eg > with TCP. But that depends on the rest of your system, and is a quite > separate question - and quite probably one you already have the answer > to.
For Json, you need to have a way of delimiting messages -- to my knowledge, Python's json library does not support parsing streams. You can send the json document in the body of a Http POST, or a ZeroMQ message, or in a UDP datagram (if you can guarantee it fits inside one) or in a simple TCP-based encapsulation mechanism that e.g. prepends the length of the message to the document. e.g. '\x00\x00\x00\x07{"a":1}' As MessagePack already does this, you can send MessagePack documents via an ordinary TCP socket and easily recover them on the other side of the pipe. >>> import msgpack; from StringIO import StringIO >>> s = StringIO(msgpack.dumps({"a":1}) + msgpack.dumps({"b":2})) >>> for doc in msgpack.Unpacker(s): ... print doc ... {'a': 1} {'b': 2} This won't work with Json: >>> import json; from StringIO import StringIO >>> s = StringIO(json.dumps({"a":1}) + json.dumps({"b":2})) >>> for doc in json.load(s): # or whatever ??? ... print doc ... Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/json/__init__.py", line 290, in load **kw) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 368, in decode raise ValueError(errmsg("Extra data", s, end, len(s))) ValueError: Extra data: line 1 column 9 - line 1 column 17 (char 8 - 16) Note that this is a limitation of python's Json parser, not Json itself. There seems to be a json.scanner module that *sounds* like it provides this functionality, but I couldn't find any documentation about it. Alternatively, PyYaml can also parse streams. yaml.{dump,load}_all() provide pickle-like unsafe (de)serialization support and yaml.safe_{dump,load}_all provide msgpack-like safe-but-limited stream parsing support. also; On 07/15/13 13:57, Chris Angelico wrote: > But what I meant was that the [Json] protocol itself is designed with > security restrictions in mind. It's designed not to fetch additional > content from the network (as XML can), Can you explain how parsing XML can fetch data from the network? Best, Burak -- http://mail.python.org/mailman/listinfo/python-list