Re: Creating JSON from RDF

[Niklas Lindström]

Hi Jeni!

2010/1/7 Jeni Tennison :
> Thanks for the pointer! The compact (natural) example that you give at [1]
> is pretty much what I'd like to see as the JSON generated by default by a
> linked data API. Have you used this in anger anywhere?

No and yes. :) That is, I haven't (yet) used this specific format, but
I designed it based on previous practical experience of accessing RDF
via (dynamic) code in various situations (including generating similar
JSON from RDF, but without all of the Gluon features). So I feel
confident it has practical value. I think it is sound to express (and
thus abstract) usage in mappings like these, rather than clutting the
code with access details over and over. That is, if the data is
expected to be fairly uniform (think "perceived vs. real dangers of
dynamic typing").

> What we're trying to do in the linked-data-api project [2] is thrash out not
> only what that JSON looks like but also how it might be generated
> automatically by a server sitting on top of a triplestore. But you say that
> you're working on a profiling mechanism? Perhaps we can join forces to work
> out what that might look like?

Yes, I would love to join forces and contribute to something within a
larger context. I think that this is a very important topic, which may
simplify many various existing practises (especially ones hitting the
complexity barriers when working directly with the full RDF model in
instrumental code contexts). Consensus on syntax and wider adoption is
crucial, so I want these things as standarized as possible.

Gluon has a "full" (raw) form, where special objects represent RDF
data concepts (using "$uri", "$ref", "$value", "$datatype", "@"
and "$list" as keys) . But the profile mechanism is there to eliminate
potentially all of these, only requiring prefix declarations (to make
the profile itself more compact). The full form still uses
qnames/CURIEs for property names though.

The "compact" form also uses a "linked" map (JSON object) where the
URI:s are used as keys. And it supports giving a global "lang" and
"base" to resolve localized literals and relative URI:s respectively.

(Note that Gluon also always inlines bnode objects, since it
(currently by opinionated design) does not support bnode-ID:s.)

What I did in the profile mechanism of Gluon follows your
linked-data-api ideas. It's originally based on my Oort "RDF to Python
object access" mechanism, where property URI:s are bound to short
names (ideally using the local part of the property URI to promote
familiarity) along with "schema"-like settings such as one/many and
localized or datatype/native. In Gluon, in order to make the JSON very
tight, I added the specific settings for interpreting plain string
values as e.g. URI:s, qnames, values with a given datatype, or a
localized language literal.

All of these features are exemplified in the compact example at [1],
which is the exact Gluon representation of the RDF (in N3) at [2].

I should mention that the python "lab" implementation of Gluon is
fully functional, capable of converting RDF data from and to this
format, taking an optional profile to produce the compact form. (Note
that it requries the currently unreleased RDFLib 2.5, i.e. from trunk
[3].)

So in essence this can work on top of a triplestore, but you need to
limit the "amount of graph" to serialize somehow. ;) It works best
when you store RDF in chunks of "reasonably" (file-sized)
representations, and convert those. In that case you only need to
design a profile for the desired JSON form (covering the expected
properties and classes).

I have thought some about generating profiles from vocabularies
(inferring them from RDFS and OWL statements), and/or by examining the
graph while serializing to "pack it on the fly". I have no code for
that yet though. There are of course alternatives to ponder more about
as well; e.g. the merit of automatically turning:

"rdfs:seeAlso": {"$ref": "http:..."}

into e.g.:

"rdfs_seeAlso_ref": "http:..."

(I don't really like that though (the abstraction starts to "bulge").
I'd rather either be fully RDF-aware with a rich object model, or use
a lowering translation "lens", such as Gluon profiles. Still, the
"prefix as suffix" has merit, as seen e.g. in python-based RDF
wrappers such as Sparta [4].)

Furthermore, I think that mapping to JSON is not the only potential
target in scope. Specifically, the design of these
mapping/context/profile features is useful in many rdf-to-object
mapping situations. And as potential input to the ideas of profiles
(short term names) in the current work on RDFa 1.1 (which Mark  knows
much more about). I honestly don't think that this would cost much
more in terms of design than to limit the scope to JSON alone. But
that is certainly up for debate.

I'd like to stress that ironing out this pattern may provide a *huge*
win, since there are *many* mapping mechanisms (in Python, Java, Ruby
etc.) for accessing RDF in programmatic contexts which a

Re: Creating JSON from RDF

[Jeni Tennison]

Hi Niklas,

On 4 Jan 2010, at 22:32, Niklas Lindström wrote:

"Gluon is a JSON format for RDF. It has a full syntax covering
properties, resource references, bnodes and literals with optional
datatype or language. With profiles, more more succinct forms are
possible."


Thanks for the pointer! The compact (natural) example that you give at  
[1] is pretty much what I'd like to see as the JSON generated by  
default by a linked data API. Have you used this in anger anywhere?


What we're trying to do in the linked-data-api project [2] is thrash  
out not only what that JSON looks like but also how it might be  
generated automatically by a server sitting on top of a triplestore.  
But you say that you're working on a profiling mechanism? Perhaps we  
can join forces to work out what that might look like?


Thanks,

Jeni

[1]: 
http://code.google.com/p/oort/source/browse/trunk/lab/gluon/etc/examples/lcsh/sh95000541-compact.json
[2]: http://code.google.com/p/linked-data-api/
--
Jeni Tennison
http://www.jenitennison.com

Re: Creating JSON from RDF

[Niklas Lindström]

Hi all!

On Thu, Dec 17, 2009 at 4:40 PM, Mike Bergman  wrote:
> Hi Dave,
>
> Dave Reynolds wrote:
>>
>> Dave Reynolds wrote:
>>
>>> Jeni Tennison wrote:
>>
 I don't know where the best place is to work on this: I guess at some
 point it would be good to set up a Wiki page or something that we could use
 as a hub for discussion?
>>>
>>> I'd suggest setting up a Google Code area and making anyone who is
>>> interested a committer. That gives us a Wiki but also hosting for associated
>>> code for generating/navigating the format. I'd be happy to set one up.
>>
>> Now done. We've opened a Google Code area "linked-data-api" and summarized
>> both the aims/assumptions [1] and the various design issues and suggestions
>> that have come out of the list discussions so far [2].
>
> Excellent work pulling this information together.
>
> Please see my comments at [1].
>
> Thanks, Mike

I've followed this with great interest! I've been tinkering with the
same things, mostly in conjunction with my Oort project [1]. I had
actually begun sketching on a JSON format I call Gluon just as this
discussion appeared (inspired by jRDF, irON, GData-in-JSON and
others). To wrap my head around it more, I've spent some time recently
to elaborate on my examples and write an initial working
implementation in Python. To pitch it:

"Gluon is a JSON format for RDF. It has a full syntax covering
properties, resource references, bnodes and literals with optional
datatype or language. With profiles, more more succinct forms are
possible."

I think that it covers many of the things you discuss here and at [2].
Of course it is an experimental design, but I hope it will be of
interest to you.

Until I've written a cohesive description at [3], I hope the specs
[4], examples [5] and implementation [6] may provide you with insight
into the design decisions I've made. Especially the compact example at
[7] describes how to make the JSON more "natural".

(Early last year I made SPARQLTree [8], which has influenced my design
somewhat and may also be interesting to you.)

Depending on how things develop, I'd be happy to contribute to [2] if
that's suitable.

(.. I'm also sketching on how to use the "profile" mechanisms to
define a language-agnostic way of accessing RDF graphs as "plain"
objects (which may update/replace my current Object-RDF mapping
solution in "Oort the python package" from which "Oort the umbrella
project" stems). Of course this also relates to e.g. the API
vocabulary and programmatic (js) access to JSON-with-references ideas
which Jeni and Mark have discussed.)

Best regards,
Niklas Lindström

[1] = 
[2] = 
[3] = 
[4] = 

[5] = 

[6] = 

[7] = 

[8] = 

Re: Creating JSON from RDF

[Mike Bergman]

Hi Dave,

Dave Reynolds wrote:

Dave Reynolds wrote:


Jeni Tennison wrote:


I don't know where the best place is to work on this: I guess at some 
point it would be good to set up a Wiki page or something that we 
could use as a hub for discussion?


I'd suggest setting up a Google Code area and making anyone who is 
interested a committer. That gives us a Wiki but also hosting for 
associated code for generating/navigating the format. I'd be happy to 
set one up.


Now done. We've opened a Google Code area "linked-data-api" and 
summarized both the aims/assumptions [1] and the various design issues 
and suggestions that have come out of the list discussions so far [2].


Excellent work pulling this information together.

Please see my comments at [1].

Thanks, Mike



The wiki is public access for reading and commenting.
Anyone who is interested in joining in with working on this please ping 
either Jeni or myself and we'll add you as a committer.


Cheers,
Dave

[1] http://code.google.com/p/linked-data-api/wiki/AssumptionsAndGoals
[2] http://code.google.com/p/linked-data-api/wiki/DesignQuestions

Re: Creating JSON from RDF

[Mark Birbeck]

An excellent summary of the state of play so far, Dave...count me in. :)


On Thu, Dec 17, 2009 at 1:48 PM, Dave Reynolds
 wrote:
> Dave Reynolds wrote:
>
>> Jeni Tennison wrote:
>
>>> I don't know where the best place is to work on this: I guess at some
>>> point it would be good to set up a Wiki page or something that we could use
>>> as a hub for discussion?
>>
>> I'd suggest setting up a Google Code area and making anyone who is
>> interested a committer. That gives us a Wiki but also hosting for associated
>> code for generating/navigating the format. I'd be happy to set one up.
>
> Now done. We've opened a Google Code area "linked-data-api" and summarized
> both the aims/assumptions [1] and the various design issues and suggestions
> that have come out of the list discussions so far [2].
>
> The wiki is public access for reading and commenting.
> Anyone who is interested in joining in with working on this please ping
> either Jeni or myself and we'll add you as a committer.
>
> Cheers,
> Dave
>
> [1] http://code.google.com/p/linked-data-api/wiki/AssumptionsAndGoals
> [2] http://code.google.com/p/linked-data-api/wiki/DesignQuestions
>

Re: Creating JSON from RDF

[Frederick Giasson]

Hi Jeni!
Thanks, that looks interesting, and the idea of supporting similar 
mappings to XML and to CSV is very attractive.

I also find this attractive :)

But I couldn't actually work out how I would use it in the kind of 
situation we find ourselves in. We have, for example, RDF like this:


  http://education.data.gov.uk/doc/school/520965.rdf

which we want to make readily usable by people creating mash-ups. What 
are the (minimal) definitions that we have to create to use irON to 
create customised JSON/CSV/XML from this RDF?


Normally, it is the other way around: people do use such notations (irON 
with its different profiles) to "intuitively" (at least, more 
intuitively) create data. So, one of the first goal of irON was to help 
data publishers to publish data without knowing and understanding all 
the specificities of RDF and all its serialization, while keeping it 
quite flexible to be able to describe complex data models. However, one 
of the requirement of irON was to be able to easily map it to RDF (when 
reading the specification, you will notice some similar concepts, which 
is a consequence of this requirement).


But naturally, if you primary source of data is in RDF, and you want to 
describe it in irON (with let sa, the irJSON profile), then can easily 
do the transformations (since RDF is probably the most flexible 
description framework out there, it is not a big deal to transform it in 
any other notation).


To make it complete, what would be required are three things:

(1) the linkage schema which describe the transformation rules
(2) the "schema" itself which describe the usage of the type/attributes 
(this is the ontology)

(3) the record instances file (which is/are the dataset files)

Lets take that RDF from your example above:


==
 rdf:about="http://education.data.gov.uk/doc/school/520965";>

   http://xmlns.com/foaf/0.1/Document"/>
   http://purl.org/dc/dcmitype/Text"/>
   rdf:resource="http://education.data.gov.uk/id/school/520965"/>

   Linked Data for Stepping Stones Nursery Group
   rdf:resource="http://education.data.gov.uk/doc/school/520965.rdf"/>
   rdf:resource="http://education.data.gov.uk/doc/school/520965.html"/>


   rdf:resource="http://education.data.gov.uk/doc/school/520965.json"/>
   rdf:resource="http://education.data.gov.uk/doc/school/520965.ttl"/>

 
==


Lets start with the linkage schema that would link the irON 
attributes/types to the RDF properties/classes



==
{
   "linkage": {
   "version": "0.10",
  
   "linkedFormat": "application/rdf+xml",
  
   "prefixList": {

   "dcterms": "http://purl.org/dc/terms/";,
   "dctypes": "http://purl.org/dc/dcmitype/";,
   "foaf": "http://xmlns.com/foaf/0.1/";
   },
  
   "typeList": {

   "Document": {
   "mapTo": "foaf:Document"
   }
   "Text": {
   "mapTo": "dctypes:Text"
   }
   },
  
   "attributeList": {

   "primaryTopic": {
   "mapTo": "foaf:primaryTopic"
   },
   "title": {
   "mapTo": "dcterms:title"
   },
   "hasFormat": {
   "mapTo": "dcterms:hasFormat"
   }
   }
   }
}
==

Note 1: you can change the name of the attributes, I used the same as 
the one of the ontology to save some time...
Note 2: I would suggest you to take a look at the Bibliographic Ontology 
for some of the "documents" description in your dataset: 
http://bibliontology.com (check the group and the ontology description).




Then you can describe a "schema" to give more information about the 
usage of these properties to the data consumers


==
{
   "schema": {
   "version": "0.10",
   "date": "17 December, 2009",



   "attributeList": {
   "title": {
   "prefLabel": "title",
   "description": "Title of a document",
  
   "allowedType": ["Document", "Text"],

   "allowedValue": {"primitive": "String"}
   },
 
   ..


   }
   }
}

==

*** As I noted in another email I send, the specification section 
related to schemas as been greatly upgraded, but we yet have to update 
the online document.




Then you can describe your record, in a dataset file, like this:


==
{
   "dataset": {
   "id": "http://education.data.gov.uk/...";,
   "prefLabel": "Dataset name",
   "description": "...",
   "source": {
   "name": "UK Government Data Developers",
   "ref": "@abc"
   },
   "creationDate": "02/09/2009",
   "linkage": "linakge_file_above.json",
   "schema": "schema_file_above.json"
   },
   "recordList": [
   {
   "id": "id_1",
   "type": "Document",
   "type": "Text",
   "title": "Linked Data for Stepping Stones Nursery Group",
   "primaryTopic": {
 

Re: Creating JSON from RDF

[Dave Reynolds]

Dave Reynolds wrote:


Jeni Tennison wrote:


I don't know where the best place is to work on this: I guess at some 
point it would be good to set up a Wiki page or something that we 
could use as a hub for discussion?


I'd suggest setting up a Google Code area and making anyone who is 
interested a committer. That gives us a Wiki but also hosting for 
associated code for generating/navigating the format. I'd be happy to 
set one up.


Now done. We've opened a Google Code area "linked-data-api" and 
summarized both the aims/assumptions [1] and the various design issues 
and suggestions that have come out of the list discussions so far [2].


The wiki is public access for reading and commenting.
Anyone who is interested in joining in with working on this please ping 
either Jeni or myself and we'll add you as a committer.


Cheers,
Dave

[1] http://code.google.com/p/linked-data-api/wiki/AssumptionsAndGoals
[2] http://code.google.com/p/linked-data-api/wiki/DesignQuestions

Re: Creating JSON from RDF

[Danny Ayers]

2009/12/14 Richard Light :
> In message , "Hammond, Tony"
>  writes
>>
>> Normal developers will always want simple.
>
> Surely what normal developers actually want are simple commands whereby data
> can be streamed in, and become available programmatically within their
> chosen development environment, without any further effort on their part?

To my mind that's very well put. But I would argue against that a
cost/benefit case - ok, it's programming hell, but it doubles your
salary - would any of us complain?


> Personally I don't see how providing a format which is easier for humans to
> read helps to achieve this.  Do normal developers like writing text parsers
> so much?

I don't know about you, but anything that helps avoiding writing
parsers is honey to me.

> Give 'em RDF and tell them to develop better toolsets ...

Why not?

> Come to that, RDF-to-JSON conversion could be a downstream service that
> someone else offers.  You don't have to do it all.

I am a bit annoyed we haven't got much pluggability between systems
yet, but strongly believe this is the right track.

Cheers,
Danny.

-- 
http://danny.ayers.name

Re: Creating JSON from RDF

[Kjetil Kjernsmo]

On Monday 14. December 2009 11:17:37 Dan Brickley wrote:
> I wish that kind of funding was easy to come
> by, but it's not. A lot of the work we need to get done around here to
> speed up progress is pretty boring stuff. It's not cutting edge
> research, nor the core of a world-changing startup, nor a good topic
> for a phd.

I can attest to that. I've been searching for this kind of funding for half 
a year now, and I've failed. The RDF toolchain is in a terrible state, but 
I do not think the focus should be on parser, and that basic stuff. Even 
though native toolchains are missing for some languages, the things that 
are there, notably Redland with all its language bindings, are still good 
enough.

What's missing are the tools that are needed on the top of that to make the 
developers efficient, not just in publishing data, but more importantly 
using the data to provide groundbreaking services that are truly useful to 
a large number of people. I've been around for 11 years, albeit more to and 
from than danbri, but I have yet to see one semweb application that does 
this. Some come close, like DBPedia Mobile, it had a lot of potential, but 
it needed polish that it hasn't got so far.

One problem, is that there are some open questions, like how to use an 
ever-changing model in an existing Model View Controller framework. This 
was something I tried to address, as it should be the strength of RDF and 
something to distinguish us from the SQL world, but the business 
perspective says that this is research and the researchers say it is mere 
stamp collecting. I've been involved in research projects that have had it 
as their main goal to provide semweb frameworks for general consumption, 
that has spent huge amounts of money on research, but has totally failed to 
achieve their goal because they didn't acknowledge that this problem, 
however trivial it may seem, has not been solved, or at least, the solution 
isn't on PEAR, CPAN or in Debian.

That brings me to another, but smaller problem, where the tools exist, with 
the exception of Redland, they are not available, and/or have not gone 
through the QA process of for example Debian. It is extremely important for 
a developer to have quick access to the toolchain. It is therefore 
critical, I believe, that e.g. Virtuoso and 4store goes into Debian.

Due to my failure to find funding, I'm changing strategy, I'm leaving the 
semweb world, hopefully temporarily, to learn the tools that good 
developers are using. I believe that we're not going forward unless we can 
provide developer tools, and they are not there. 

Cheers,

Kjetil
-- 
Kjetil Kjernsmo
kje...@kjernsmo.net
http://www.kjetil.kjernsmo.net/

Re: Creating JSON from RDF

[Dave Reynolds]

Mark Birbeck wrote:


On Sat, Dec 12, 2009 at 9:42 PM, Jeni Tennison  wrote:



One thing that we want to do is provide JSON representations of both RDF
graphs and SPARQL results. I wanted to run some ideas past this group as to
how we might do that.


Great again. :)

In the work I've been doing, I've concluded that in JSON-world, an RDF
graph should be a JSON object (as explained in RDFj [1], and as you
seem to concur), but also that SPARQL queries should return RDFj
objects too.

In other words, after a lot of playing around I concluded that there
was nothing to be gained from differentiating between representations
of graphs, and the results of queries.


OK but the relationship between the JSON objects you get back from 
queries and the source RDF graph may be indirect. In a SPARQL query you 
sometimes pull pieces out of various depths of connected objects but 
from the JSON consumer you probably want to think of the results as an 
array of simple structures.



Note that the "$" is taken from RDFj. I'm not convinced it's a good idea to
use this symbol, rather than simply a property called "about" or "this" --
any opinions?


I agree, and in my RDFj description I do say that since '$' is used in
a lot of Ajax libraries, I should find something else.

However, in my view, the 'something else' shouldn't look like a
predicate, so I don't think 'about' or 'this' (or 'id' as someone
suggests later in the thread), should be used. (Note also that 'id' is
used in a related but slightly different way by Dojo.)

Also, the underscore is generally related to bnodes, so it might be
confusing on quick reads through. (We have a JSON audience and an RDF
audience, and need to make design decisions with both in mind.)

I've often thought about the empty string, '@' and other
possibilities, but haven't had a chance to try them out. E.g., the
empty string would simply look like this:

  {
"": "http://www.w3.org/TR/rdf-syntax-grammar";,
  "title": "RDF/XML Syntax Specification (Revised)",
  "editor": {
"name": "Dave Beckett",
"homepage": "http://purl.org/net/dajobe/";
  }
  }

Since I always tend to indent the predicates in RDFj anyway, just to
draw attention to them, then the empty string is reasonably visible.
However, "@" would be even more obvious:

  {
"@": "http://www.w3.org/TR/rdf-syntax-grammar";,
  "title": "RDF/XML Syntax Specification (Revised)",
  "editor": {
"name": "Dave Beckett",
"homepage": "http://purl.org/net/dajobe/";
  }
  }

Anyway, it shouldn't be that difficult to come up with something.


Naming discussions are often the hardest and can be non-terminating :) 
there is never only one acceptable answer, rarely a really good answer 
and everyone has opinions. Syntax and semantics are much easier than names.


As I've said, I like "id" as in freebase, I'd go along with "_about" 
(thought I take your point about bNode confusion) and I'd go along with 
"@" (but it makes me think of pointers rather than ids).



So, the first piece of configuration that I think we need here is to map
properties on to short names...


That's what 'tokens' in the 'context' object do, in RDFj.


I think there's two separate things here. How the producer of the JSON 
maps their RDF to JSON names and then whether the consumer is able to 
inspect that mapping.


For the first of those, I think the proposal is that there be a set of 
default conventions (e.g. as in Exhibit JSON) plus an optional mapping 
spec which enables the names to be improved.


For the second part, I agree with you than a context object in the 
delivered JSON would be useful so that those consumers who care can see 
the mapping what was applied and even invert it.



However, in any particular graph, there may be properties that have been
given the same JSON name (or, even more probably, local name). We could
provide multiple alternative names that could be chosen between, but any
mapping to JSON is going to need to give consistent results across a given
dataset for people to rely on it as an API, and that means the mapping can't
be based on what's present in the data. We could do something with prefixes,
but I have a strong aversion to assuming global prefixes.


I'm not sure here whether the goal is to map /any/ API to RDF, but if
it is I think that's a separate problem to the 'JSON as RDF' question.


I think the problem is getting dev-friendly 'JSON out of RDF' rather 
than 'JSON as RDF', being able to invert that get back the RDF would be 
an added bonus rather than design requirement.



## Multi-valued Properties ##

First one first. It seems obvious that if you have a property with multiple
values, it should turn into a JSON array structure. For example:

 [] foaf:name "Anna Wilder" ;
   foaf:nick "wilding", "wilda" ;
   foaf:homepage  .

should become something like:

 {
   "name": "Anna Wilder",
   "nick": [ "wilding", "wilda" ],
   "homepage": "http://exam

Re: Creating JSON from RDF

[Dave Reynolds]

Hi Jenni,

Jeni Tennison wrote:


On 13 Dec 2009, at 13:34, Dave Reynolds wrote:


I agree we want both graphs and SPARQL results but I think there is 
another third case - lists of described objects.


I absolutely agree with you that lists of described objects is an 
essential part of an API. In fact, I was going to (and will!) write a 
separate message about possible approaches for creating such lists.


It seemed to me that lists could be represented with RDF like:

  
rdfs:label "Local Authorities - Page 1" ;
xhv:next  ;
...
api:contents (
  
  
  
  ...
)


This is just RDF, and as such any rules that we create about mapping RDF 
graphs to JSON could apply. (I agree that the list page should include 
extra information about the items in the list, but that seems to me to 
be a separable issue.)


Sure but there are some advantages to treating this ordered list of 
results as an API issue rather than a modelling issue.


I'll respond properly on your other thread.

One thing it makes me think is that perhaps JSON Schema [1] could form 
the basis of the mechanism for expressing any extra stuff that's 
required about the properties.


Interesting thought, I'll need to go learn more about JSON Schema first.

Note that the "$" is taken from RDFj. I'm not convinced it's a good 
idea to use this symbol, rather than simply a property called "about" 
or "this" -- any opinions?


I'd prefer "id" (though "about" is OK), "$" is too heavily overused in 
javascript libraries.


I agree. From the brief survey of JSON APIs that I did just now, it 
seems as though prefixing a reserved property name with a '_' is the 
usual thing. I'd suggest '_about' because it's similar to RDFa and 
because '_id', to me at least, implies a local identifier rather than a 
URI.


No objection to "_about", as per separate thread it was Freebase 
especially that motivated the suggestion of "id".


[On api:mapping usage]
Are you thinking of this as something the publisher provides or the 
API caller provides?


If the former, then OK but as I say I think a zero config set of 
default conventions is OK with the API to allow fine tuning.


I'm thinking of this as something that the publisher of the API creates 
(to describe/define the API). Note, though, that the publisher of the 
API might not be the publisher of the data, and that it could feasibly 
be possible for there to be a service that would allow clients to supply 
a configuration, point at a datastore, and have the API just work.


OK, agreed. My concern is that developers shouldn't have to wade through 
this mapping to understand what they are getting, unless they are 
already RDF heads and care about that aspect.


[On multi-valued properties]

I guess there are two choices if there was no specification:

  1. always give one value for the property; if there are several values 
in the graph, then provide "the first"
  2. give an array when there are multiple values and a singleton when 
there's only one


I did have another vague notion of providing two properties side by 
side, one singular and one plural, so you would have:


  {
"nick": "JeniT"
  }

or

  {
"nicks": ["wilding", "wilda"]
  }

side by side in the same list of objects. But of course that would 
require configuration anyway (to provide pluralised versions of the 
label), so I'm not particularly taken with it.


It does concern me that if there are RDF graphs which contain 
descriptions of several resources of the same type, we might get into a 
situation where there are two resources for which the default behaviour 
would be different; we need to have a way of reconciling this (for 
example, if any of the resources in the graph have multiple values for a 
property, then it always uses an array).


Yes. With zero configuration there will always either be some 
inconsistency or you have to force the more general convention on 
people. I agree with Mark that developers can write code to adapt to the 
list/no-list case and with configuration we have the option to make this 
more consistent in places where this is a problem.


One possibility is a bootstrapping service where you give sample data 
and ontology, if available, and get back suggested mapping. That can do 
the scanning of data to guess at multi-valuedness once so you don't pay 
the cost of doing that in the live API.



[snip]
Language codes are effectively open ended. I can't necessarily predict 
what lang codes are going to be in my data and provide a property 
mapping for every single one.


I know they're *potentially* open-ended; I think in practice, for a 
single API, they are probably not. 


Depends on whether this is your own d

Re: Creating JSON from RDF

[Dave Reynolds]

Jeni Tennison wrote:

It's worth noting that most of these APIs support a callback= parameter 
that makes the API return Javascript containing a function call rather 
than simply the JSON itself. I regard this as an unquestionably 
essential part of a JSON API, whether it uses RDF/JSON or RDFj or irJSON 
or whatever, in order to make the JSON usable cross-site.


+1

Dave

Re: Creating JSON from RDF

[Dave Reynolds]

Jeni Tennison wrote:

On 12 Dec 2009, at 22:27, Danny Ayers wrote:

I can't offer any real practical suggestions right away (a lot to
digest here!) but one question I think right away may some
significance: you want this to be friendly to normal developers - what
kind of things are they actually used to? Do you have any examples of
existing JSON serializations of relatively complex data structures -
something to give an idea of the target.

While it's a bit apples and oranges, there presumably are plenty of
systems now pushing out JSON rather than the XML they would a few
years ago - is there anything to be learnt from that scenario that can
be exploited for RDF?


Great idea to look at what currently exists. Let's see.

The flickr API [1] is notable in that the JSON is a direct mapping from 
the XML API. From what I can tell they don't even try to use values that 
are anything other than strings, and they have a moderately ugly 
"_content" property for when elements in the XML API have content 
(though it mostly uses attributes, from what I can tell).


The Twitter API [2] provides JSON and XML (Atom) formats. There are a 
whole bunch of different queries you can do, most of which contain 
nested objects etc. Interestingly, the JSON that you get back from a 
search does contain metadata about the search and a 'results' property 
that holds an array of the results. So that could be a relevant model 
for SPARQL results formats.


CouchDB [3] is a purely JSON API which has to be very generic (since 
CouchDB can be used to contain anything). It uses reserved property 
names like "_id" (this and "_content" in the flickr API make me think 
that a leading underscore is the expected way to create reserved 
property names).


Yahoo! [4] have a JSON API that is again based on an XML API with a 
straight-forward mapping from the XML to the JSON.


The Google Data Protocol [5] uses JSON that is generated from the 
equivalent Atom feed. Interestingly, they provide support for namespaces 
by having "xmlns" properties (with "$" used instead of ":" in any 
qualified names). Unlike the other APIs, they do use namespaces, but 
only a handful. I strongly doubt that any developer using it would 
actually resolve "openSearch$startIndex" by looking at the 
"xmlns$openSearch" property.


Is that enough of an idea to be getting on with?


Probably, though one additional particularly relevant one is Freebase. 
That has a somewhat RDF-like abstract data model but data is both 
queried and returned as JSON. A query is a JSON template with nulls and 
blank structures where you want details filled in (plus meta properties 
for operations like sorting). Everything is identified by an "id" (a 
Freebase topic path, pretty much the trailing part of a URI) and/or a 
"guid". Freebase is one reason I suggested "id" as a relatively familiar 
property label for identifiers.


Cheers,
Dave

[1] http://www.freebase.com/docs/data
[2] http://www.freebase.com/docs/data/first_query

Re: Creating JSON from RDF

[Richard Light]

In message 
, Dan 
Brickley  writes



With every passing year the RDF tools do get a bit better, but also
the old ones code rot a bit, or new things come along that need
supporting (GRDDL, RDFa etc.). What can be done in the SemWeb and
Linked Data scene so that it becomes a bigger part of people's real
dayjobs to improve our core tooling? Are the resources already out
there but poorly coordinated? Would some lightweight collective
project management help? Are there things (eg. finalising a ruby
parser toolkit) that are weekend-sized jobs, month sized jobs; do they
look more like msc student summer projects or EU STREP / IP projects
in scale? Could we do more by simply transliterating code between
languages? ie. if something exists in Python it can be converted to
Ruby or vice-versa...? Are funded grants available  (eg. JISC in UK?)
that would help polish, package, test and integrate basic entry-level
RDF / linked data software tools?

Back on the original thread, I am talking here so far only about core
RDF tools, eg. having basic RDF -to- triples facility available
reliably in some language of choice. As Jeni emphasises, there are
lots of other pieces of bridging technology needed (eg. into modern
JSON idioms). But when we are hoping to convert folk to use pure
generic RDF tools, we better make sure they're in good shape. Some
are, some aren't, and that lumpy experience can easily turn people
away...


I think some lightweight collective documentation work would help. 
There is nothing I can spot on RDF or Linked Data software tools in the 
W3C pages [1].  The most recent entry in the Editors and Tools section 
of Dave Beckett's RDF Resource Guide [1] is dated 2005.  Robin Cover 
defers to this resource and has added nothing since 2004.  Am I missing 
a key resource here?


If we produced a grid, with RDF/Linked Data tasks on one axis and 
programming languages on the other, we could at least start to map the 
space.


Richard

[1] http://www.w3.org/2001/sw/
[2] http://planetrdf.com/guide/
--
Richard Light

Re: Creating JSON from RDF

[Mark Birbeck]

Hi Richard,

> Despite the "Bah, Humbug!" tone of my previous mail, I am actually in favour
> too.  I just want to tease out the extent to which we would be giving
> developers what they say they want, rather than what they could actually
> use.
>
> The value of JSON is surely that JSON support is widespread across a wide
> range of programming environments[1], rather than that JSON is a "simple"
> format.  Would the alternative formats being discussed offer as much, in
> terms of their practical implementation for a wide range of languages?

I'm speaking for myself here, but going on her first email, I'm pretty
sure that Jeni is taking the same approach.

The goal is to make it easy for web programmers to get RDF data
directly into their applications. So although it's true that JSON is a
general-purpose format, our interest here is more specific, in that it
easy to get JSON data directly into web applications.

For those who don't do a lot of JavaScript programming, I should point
out a subtlety here that might be being missed; although modern web
apps make extensive use of XMLHttpRequest objects to retrieve data, in
ordinary circumstances it is not possible to use XHR to get data from
a different domain to the one hosting HTML document.

However, HTML /does/ allow the 

Re: Creating JSON from RDF

[Richard Light]

In message 
, Dan 
Brickley  writes



RDF tooling still has some rough edges, it must be said. I am as
enthusiastic about RDF as anyone (having been involved since 1997) but
I've also seen the predictable results where on occasion people (eg.
standards groups) have been 'arm twisted' into using the technology
against their judgement and preferences. We don't have a solid
well-packaged and tested RDF/XML parser for the Ruby language yet, for
example. And while we do have librdfa integration into the
Redland/Raptor C toolkit, it hasn't yet propagated into all the easy
install settings we'll eventually find it - like my Amazon EC2 Ubuntu
box, or the copy of Fink I installed recently on my MacBook Pro. And
in PHP we have a fantastic RDF toolkit in ARC2, but it relies on MySQL
for all complex querying. Plenty of scope for toolkit polish and
improvement, nothing to worry massively about, but also lots of things
that will cause pain if we take a stubborn "RDF or nothing" approach.
I wholeheartedly applaud the pragmatic approach from Jeni and others.


Despite the "Bah, Humbug!" tone of my previous mail, I am actually in 
favour too.  I just want to tease out the extent to which we would be 
giving developers what they say they want, rather than what they could 
actually use.


The value of JSON is surely that JSON support is widespread across a 
wide range of programming environments[1], rather than that JSON is a 
"simple" format.  Would the alternative formats being discussed offer as 
much, in terms of their practical implementation for a wide range of 
languages?



Come to that, RDF-to-JSON conversion could be a downstream service that
someone else offers.  You don't have to do it all.


That could be useful for some, and inappropriate for others. Every new
step in the chain introduces potential problems with latency, bugs,
security and so on...


I was thinking of a dynamic web service, which would at least remove the 
latency issue.


Richard

[1] http://json.org/
--
Richard Light

Re: Creating JSON from RDF

[Mark Birbeck]

Hi Richard,

Isn't:

> Give 'em RDF and tell them to develop better toolsets ...

exactly the approach that the RDF community has been pursuing for a
decade or two, with less than impressive results?

;)

Regards,

Mark

--
Mark Birbeck, webBackplane

mark.birb...@webbackplane.com

http://webBackplane.com/mark-birbeck

webBackplane is a trading name of Backplane Ltd. (company number
05972288, registered office: 2nd Floor, 69/85 Tabernacle Street,
London, EC2A 4RR)

Re: Creating JSON from RDF [OpenSearch Groups Link]

[Hammond, Tony]

Hi Damian:

Sorry 'bout that. I did indeed include the wrong link (I was logged in at at
the time). DeWitt's mail on the OpenSearch list is here:

http://groups.google.com/group/opensearch/msg/f2f724e0e3123150

Here he's advocating JSONC as a possible (and practical) way forward.

Cheers,

Tony


On 14/12/09 10:13, "Damian Steer"  wrote:

> http://mail.google.com/mail/h/11f0xqx6m4lpr/?v=c&th=1256fa3bf3f1c75f



   
DISCLAIMER: This e-mail is confidential and should not be used by anyone who is
not the original intended recipient. If you have received this e-mail in error
please inform the sender and delete it from your mailbox or any other storage
mechanism. Neither Macmillan Publishers Limited nor any of its agents accept
liability for any statements made which are clearly the sender's own and not
expressly made on behalf of Macmillan Publishers Limited or one of its agents.
Please note that neither Macmillan Publishers Limited nor any of its agents
accept any responsibility for viruses that may be contained in this e-mail or
its attachments and it is your responsibility to scan the e-mail and 
attachments (if any). No contracts may be concluded on behalf of Macmillan 
Publishers Limited or its agents by means of e-mail communication. Macmillan 
Publishers Limited Registered in England and Wales with registered number 
785998 
Registered Office Brunel Road, Houndmills, Basingstoke RG21 6XS   

Re: Creating JSON from RDF

[Dan Brickley]

On Mon, Dec 14, 2009 at 10:37 AM, Jeni Tennison  wrote:
> Richard,
>
> My opinion, based on the reactions that I've seen from enthusiastic,
> hard-working developers who just want to get things done, is that we (the
> data.gov.uk project in particular, linked data in general) are not providing
> them what they need.
>
> We can sit around and wait for other people to provide the simple,
> light-weight interfaces that those developers demand, or we can do it
> ourselves. I can predict with near certainty that if we do not do it
> ourselves, these developers will not use the linked data that we produce:
> they will download the original source data which is also being made
> available to them, and use that.
>
> We, here, on this list, understand the potential power of using linked data.
> The developers who want to use the data don't. (And the publishers producing
> the data don't.) We simply can't say "but they can just build tools", "they
> can just use SPARQL". They are not going to build bridges to us. We have to
> build bridges to them.
>
> My opinion.

Opinion, sure. But absolutely correct, also!

(Excuse me if a small rant is triggered by all this...)

Why, twelve years, two months and twelve days after
http://www.w3.org/TR/WD-rdf-syntax-971002/ was first published, do we
not have well packaged, maintained and fully compliant RDF parsers
available in every major programming language?  And that is for just
the smallest critical piece of software needed to do anything useful.


Short answer: because people from these mailing lists didn't sit down
and do the work. We waited for someone else to do it. Some of us did
bits of it, but ... taken as a whole, there are still plenty of basic
pieces unfinished, in various languages.

Millions upon millions of euros and dollars have been spent on
Semantic this and Semantic that, and now Linked this and Linked that;
countless conferences, workshops and seminars, PDFs, PPTs and so on;
but still such basic software components haven't been finished,
polished, tested and distributed.

I'm not speaking ill of anyone in particular here. Countless folk have
worked hard and tirelessly to progress the state of the art, get tools
matured and deployed. But there is plenty plenty more to do. I do fear
that the structure of both academic and research (eg. EU) funding
doesn't favour the kind of work and workplan we need. In the
SWAD-Europe EU project we were very unusual to have explicit funding
and plans that allowed - for example - Dave Beckett to work not only
on the RDF Core standards, but on their opensource implementation in
C; or Jan Grant and Dave to work on the RDF Test Cases, or Alistair
Miles to take SKOS from a rough idea to something that's shaking up
the whole library world. I wish that kind of funding was easy to come
by, but it's not. A lot of the work we need to get done around here to
speed up progress is pretty boring stuff. It's not cutting edge
research, nor the core of a world-changing startup, nor a good topic
for a phd.

With every passing year the RDF tools do get a bit better, but also
the old ones code rot a bit, or new things come along that need
supporting (GRDDL, RDFa etc.). What can be done in the SemWeb and
Linked Data scene so that it becomes a bigger part of people's real
dayjobs to improve our core tooling? Are the resources already out
there but poorly coordinated? Would some lightweight collective
project management help? Are there things (eg. finalising a ruby
parser toolkit) that are weekend-sized jobs, month sized jobs; do they
look more like msc student summer projects or EU STREP / IP projects
in scale? Could we do more by simply transliterating code between
languages? ie. if something exists in Python it can be converted to
Ruby or vice-versa...? Are funded grants available  (eg. JISC in UK?)
that would help polish, package, test and integrate basic entry-level
RDF / linked data software tools?

Back on the original thread, I am talking here so far only about core
RDF tools, eg. having basic RDF -to- triples facility available
reliably in some language of choice. As Jeni emphasises, there are
lots of other pieces of bridging technology needed (eg. into modern
JSON idioms). But when we are hoping to convert folk to use pure
generic RDF tools, we better make sure they're in good shape. Some
are, some aren't, and that lumpy experience can easily turn people
away...

cheers,

Dan

Re: Creating JSON from RDF

[Dan Brickley]

On Mon, Dec 14, 2009 at 10:23 AM, Richard Light
 wrote:
> In message , "Hammond, Tony"
>  writes
>>
>> Normal developers will always want simple.
>
> Surely what normal developers actually want are simple commands whereby data
> can be streamed in, and become available programmatically within their
> chosen development environment, without any further effort on their part?
>
> Personally I don't see how providing a format which is easier for humans to
> read helps to achieve this.  Do normal developers like writing text parsers
> so much?
>
> Give 'em RDF and tell them to develop better toolsets ...

RDF tooling still has some rough edges, it must be said. I am as
enthusiastic about RDF as anyone (having been involved since 1997) but
I've also seen the predictable results where on occasion people (eg.
standards groups) have been 'arm twisted' into using the technology
against their judgement and preferences. We don't have a solid
well-packaged and tested RDF/XML parser for the Ruby language yet, for
example. And while we do have librdfa integration into the
Redland/Raptor C toolkit, it hasn't yet propagated into all the easy
install settings we'll eventually find it - like my Amazon EC2 Ubuntu
box, or the copy of Fink I installed recently on my MacBook Pro. And
in PHP we have a fantastic RDF toolkit in ARC2, but it relies on MySQL
for all complex querying. Plenty of scope for toolkit polish and
improvement, nothing to worry massively about, but also lots of things
that will cause pain if we take a stubborn "RDF or nothing" approach.
I wholeheartedly applaud the pragmatic approach from Jeni and others.

> Come to that, RDF-to-JSON conversion could be a downstream service that
> someone else offers.  You don't have to do it all.

That could be useful for some, and inappropriate for others. Every new
step in the chain introduces potential problems with latency, bugs,
security and so on...

cheers,

Dan

Re: Creating JSON from RDF

[Jeni Tennison]

Richard,

My opinion, based on the reactions that I've seen from enthusiastic,  
hard-working developers who just want to get things done, is that we  
(the data.gov.uk project in particular, linked data in general) are  
not providing them what they need.


We can sit around and wait for other people to provide the simple,  
light-weight interfaces that those developers demand, or we can do it  
ourselves. I can predict with near certainty that if we do not do it  
ourselves, these developers will not use the linked data that we  
produce: they will download the original source data which is also  
being made available to them, and use that.


We, here, on this list, understand the potential power of using linked  
data. The developers who want to use the data don't. (And the  
publishers producing the data don't.) We simply can't say "but they  
can just build tools", "they can just use SPARQL". They are not going  
to build bridges to us. We have to build bridges to them.


My opinion.

Jeni

On 14 Dec 2009, at 09:23, Richard Light wrote:

In message , "Hammond, Tony" > writes


Normal developers will always want simple.


Surely what normal developers actually want are simple commands  
whereby data can be streamed in, and become available  
programmatically within their chosen development environment,  
without any further effort on their part?


Personally I don't see how providing a format which is easier for  
humans to read helps to achieve this.  Do normal developers like  
writing text parsers so much?


Give 'em RDF and tell them to develop better toolsets ...

Come to that, RDF-to-JSON conversion could be a downstream service  
that someone else offers.  You don't have to do it all.


Richard


On 12/12/09 21:42, "Jeni Tennison"  wrote:


Hi,

As part of the linked data work the UK government is doing, we're
looking at how to use the linked data that we have as the basis of
APIs that are readily usable by developers who really don't want to
learn about RDF or SPARQL.

One thing that we want to do is provide JSON representations of both
RDF graphs and SPARQL results. I wanted to run some ideas past this
group as to how we might do that.

To put this in context, what I think we should aim for is a pure
publishing format that is optimised for approachability for normal
developers, *not* an interchange format. RDF/JSON [1] and the SPARQL
results JSON format [2] aren't entirely satisfactory as far as I'm
concerned because of the way the objects of statements are  
represented
as JSON objects rather than as simple values. I still think we  
should

produce them (to wean people on to, and for those using more generic
tools), but I'd like to think about producing something that is a  
bit

more immediately approachable too.

RDFj [3] is closer to what I think is needed here. However, I don't
think there's a need for setting 'context' given I'm not aiming  
for an
interchange format, there are no clear rules about how to generate  
it

from an arbitrary graph (basically there can't be without some
additional configuration) and it's not clear how to deal with
datatypes or languages.

I suppose my first question is whether there are any other JSON- 
based
formats that we should be aware of, that we could use or borrow  
ideas

from?

Assuming there aren't, I wanted to discuss what generic rules we  
might
use, where configuration is necessary and how the configuration  
might

be done.

# RDF Graphs #

Let's take as an example:

  
dc:title "RDF/XML Syntax Specification (Revised)" ;
ex:editor [
  ex:fullName "Dave Beckett" ;
  ex:homePage  ;
] .

In JSON, I think we'd like to create something like:

  {
"$": "http://www.w3.org/TR/rdf-syntax-grammar";,
"title": "RDF/XML Syntax Specification (Revised)",
"editor": {
  "name": "Dave Beckett",
  "homepage": "http://purl.org/net/dajobe/";
}
  }

Note that the "$" is taken from RDFj. I'm not convinced it's a good
idea to use this symbol, rather than simply a property called  
"about"

or "this" -- any opinions?

Also note that I've made no distinction in the above between a URI  
and

a literal, while RDFj uses <>s around literals. My feeling is that
normal developers really don't care about the distinction between a
URI literal and a pointer to a resource, and that they will base the
treatment of the value of a property on the (name of) the property
itself.

So, the first piece of configuration that I think we need here is to
map properties on to short names that make good JSON identifiers (ie
name tokens without hyphens). Given that properties normally have
lowercaseCamelCase local names, it should be possible to use that  
as a
default. If you need something more readable, though, it seems  
like it

should be possible to use a property of the property, such as:

  ex:fullName api:jsonName "name" .
  ex:homePage api:jsonName "homepage" .

Howeve

Re: Creating JSON from RDF

[Richard Light]

In message , "Hammond, Tony" 
 writes


Normal developers will always want simple.


Surely what normal developers actually want are simple commands whereby 
data can be streamed in, and become available programmatically within 
their chosen development environment, without any further effort on 
their part?


Personally I don't see how providing a format which is easier for humans 
to read helps to achieve this.  Do normal developers like writing text 
parsers so much?


Give 'em RDF and tell them to develop better toolsets ...

Come to that, RDF-to-JSON conversion could be a downstream service that 
someone else offers.  You don't have to do it all.


Richard


On 12/12/09 21:42, "Jeni Tennison"  wrote:


Hi,

As part of the linked data work the UK government is doing, we're
looking at how to use the linked data that we have as the basis of
APIs that are readily usable by developers who really don't want to
learn about RDF or SPARQL.

One thing that we want to do is provide JSON representations of both
RDF graphs and SPARQL results. I wanted to run some ideas past this
group as to how we might do that.

To put this in context, what I think we should aim for is a pure
publishing format that is optimised for approachability for normal
developers, *not* an interchange format. RDF/JSON [1] and the SPARQL
results JSON format [2] aren't entirely satisfactory as far as I'm
concerned because of the way the objects of statements are represented
as JSON objects rather than as simple values. I still think we should
produce them (to wean people on to, and for those using more generic
tools), but I'd like to think about producing something that is a bit
more immediately approachable too.

RDFj [3] is closer to what I think is needed here. However, I don't
think there's a need for setting 'context' given I'm not aiming for an
interchange format, there are no clear rules about how to generate it
from an arbitrary graph (basically there can't be without some
additional configuration) and it's not clear how to deal with
datatypes or languages.

I suppose my first question is whether there are any other JSON-based
formats that we should be aware of, that we could use or borrow ideas
from?

Assuming there aren't, I wanted to discuss what generic rules we might
use, where configuration is necessary and how the configuration might
be done.

# RDF Graphs #

Let's take as an example:

   
 dc:title "RDF/XML Syntax Specification (Revised)" ;
 ex:editor [
   ex:fullName "Dave Beckett" ;
   ex:homePage  ;
 ] .

In JSON, I think we'd like to create something like:

   {
 "$": "http://www.w3.org/TR/rdf-syntax-grammar";,
 "title": "RDF/XML Syntax Specification (Revised)",
 "editor": {
   "name": "Dave Beckett",
   "homepage": "http://purl.org/net/dajobe/";
 }
   }

Note that the "$" is taken from RDFj. I'm not convinced it's a good
idea to use this symbol, rather than simply a property called "about"
or "this" -- any opinions?

Also note that I've made no distinction in the above between a URI and
a literal, while RDFj uses <>s around literals. My feeling is that
normal developers really don't care about the distinction between a
URI literal and a pointer to a resource, and that they will base the
treatment of the value of a property on the (name of) the property
itself.

So, the first piece of configuration that I think we need here is to
map properties on to short names that make good JSON identifiers (ie
name tokens without hyphens). Given that properties normally have
lowercaseCamelCase local names, it should be possible to use that as a
default. If you need something more readable, though, it seems like it
should be possible to use a property of the property, such as:

   ex:fullName api:jsonName "name" .
   ex:homePage api:jsonName "homepage" .

However, in any particular graph, there may be properties that have
been given the same JSON name (or, even more probably, local name). We
could provide multiple alternative names that could be chosen between,
but any mapping to JSON is going to need to give consistent results
across a given dataset for people to rely on it as an API, and that
means the mapping can't be based on what's present in the data. We
could do something with prefixes, but I have a strong aversion to
assuming global prefixes.

So I think this means that we need to provide configuration at an API
level rather than at a global level: something that can be used
consistently across a particular API to determine the token that's
used for a given property. For example:

   <> a api:JSON ;
 api:mapping [
   api:property ex:fullName ;
   api:name "name" ;
 ] , [
   api:property ex:homePage ;
   api:name "homepage" ;
 ] .

There are four more areas where I think there's configuration we need
to think about:

   * multi-valued properties
   * typed and language-specific values
   * 

Re: Creating JSON from RDF

[Hammond, Tony]

(Sorry for my last post. That'll learn  me for composing in place.
Hamfistedly hit the wrong buttons.)

Hi Jeni:

Agree with this:

> a normal developer would want to just get:
> 
>[{
>  "book": "http://example.org/book/book6";,
>  "title": "Harry Potter and the Half-Blood Prince"
> },{

Wondered if you'd seen the JSONC proposal coming from the Google Data API
work:

http://code.google.com/events/io/2009/sessions/EvolutionGoogleDataProtocol.h
tml

This was raised by DeWitt Clinton on the Google Groups OpenSearch list which
is looking for a suitable (simple) JSON representation:

http://mail.google.com/mail/h/11f0xqx6m4lpr/?v=c&th=1256fa3bf3f1c75f

Normal developers will always want simple.

Cheers,

Tony


On 12/12/09 21:42, "Jeni Tennison"  wrote:

> Hi,
> 
> As part of the linked data work the UK government is doing, we're
> looking at how to use the linked data that we have as the basis of
> APIs that are readily usable by developers who really don't want to
> learn about RDF or SPARQL.
> 
> One thing that we want to do is provide JSON representations of both
> RDF graphs and SPARQL results. I wanted to run some ideas past this
> group as to how we might do that.
> 
> To put this in context, what I think we should aim for is a pure
> publishing format that is optimised for approachability for normal
> developers, *not* an interchange format. RDF/JSON [1] and the SPARQL
> results JSON format [2] aren't entirely satisfactory as far as I'm
> concerned because of the way the objects of statements are represented
> as JSON objects rather than as simple values. I still think we should
> produce them (to wean people on to, and for those using more generic
> tools), but I'd like to think about producing something that is a bit
> more immediately approachable too.
> 
> RDFj [3] is closer to what I think is needed here. However, I don't
> think there's a need for setting 'context' given I'm not aiming for an
> interchange format, there are no clear rules about how to generate it
> from an arbitrary graph (basically there can't be without some
> additional configuration) and it's not clear how to deal with
> datatypes or languages.
> 
> I suppose my first question is whether there are any other JSON-based
> formats that we should be aware of, that we could use or borrow ideas
> from?
> 
> Assuming there aren't, I wanted to discuss what generic rules we might
> use, where configuration is necessary and how the configuration might
> be done.
> 
> # RDF Graphs #
> 
> Let's take as an example:
> 
>
>  dc:title "RDF/XML Syntax Specification (Revised)" ;
>  ex:editor [
>ex:fullName "Dave Beckett" ;
>ex:homePage  ;
>  ] .
> 
> In JSON, I think we'd like to create something like:
> 
>{
>  "$": "http://www.w3.org/TR/rdf-syntax-grammar";,
>  "title": "RDF/XML Syntax Specification (Revised)",
>  "editor": {
>"name": "Dave Beckett",
>"homepage": "http://purl.org/net/dajobe/";
>  }
>}
> 
> Note that the "$" is taken from RDFj. I'm not convinced it's a good
> idea to use this symbol, rather than simply a property called "about"
> or "this" -- any opinions?
> 
> Also note that I've made no distinction in the above between a URI and
> a literal, while RDFj uses <>s around literals. My feeling is that
> normal developers really don't care about the distinction between a
> URI literal and a pointer to a resource, and that they will base the
> treatment of the value of a property on the (name of) the property
> itself.
> 
> So, the first piece of configuration that I think we need here is to
> map properties on to short names that make good JSON identifiers (ie
> name tokens without hyphens). Given that properties normally have
> lowercaseCamelCase local names, it should be possible to use that as a
> default. If you need something more readable, though, it seems like it
> should be possible to use a property of the property, such as:
> 
>ex:fullName api:jsonName "name" .
>ex:homePage api:jsonName "homepage" .
> 
> However, in any particular graph, there may be properties that have
> been given the same JSON name (or, even more probably, local name). We
> could provide multiple alternative names that could be chosen between,
> but any mapping to JSON is going to need to give consistent results
> across a given dataset for people to rely on it as an API, and that
> means the mapping can't be based on what's present in the data. We
> could do something with prefixes, but I have a strong aversion to
> assuming global prefixes.
> 
> So I think this means that we need to provide configuration at an API
> level rather than at a global level: something that can be used
> consistently across a particular API to determine the token that's
> used for a given property. For example:
> 
><> a api:JSON ;
>  api:mapping [
>api:property ex:fullName ;
>api:name "

Re: Creating JSON from RDF

[Hammond, Tony]

Hi Jeni:

Agree with this:

> a normal developer would want to just get:
> 
>[{
>  "book": "http://example.org/book/book6";,
>  "title": "Harry Potter and the Half-Blood Prince"
> },{

Wondered if you'd seen the JSONC proposal coming from the Google Data API
work:





> Hi,
> 
> As part of the linked data work the UK government is doing, we're
> looking at how to use the linked data that we have as the basis of
> APIs that are readily usable by developers who really don't want to
> learn about RDF or SPARQL.
> 
> One thing that we want to do is provide JSON representations of both
> RDF graphs and SPARQL results. I wanted to run some ideas past this
> group as to how we might do that.
> 
> To put this in context, what I think we should aim for is a pure
> publishing format that is optimised for approachability for normal
> developers, *not* an interchange format. RDF/JSON [1] and the SPARQL
> results JSON format [2] aren't entirely satisfactory as far as I'm
> concerned because of the way the objects of statements are represented
> as JSON objects rather than as simple values. I still think we should
> produce them (to wean people on to, and for those using more generic
> tools), but I'd like to think about producing something that is a bit
> more immediately approachable too.
> 
> RDFj [3] is closer to what I think is needed here. However, I don't
> think there's a need for setting 'context' given I'm not aiming for an
> interchange format, there are no clear rules about how to generate it
> from an arbitrary graph (basically there can't be without some
> additional configuration) and it's not clear how to deal with
> datatypes or languages.
> 
> I suppose my first question is whether there are any other JSON-based
> formats that we should be aware of, that we could use or borrow ideas
> from?
> 
> Assuming there aren't, I wanted to discuss what generic rules we might
> use, where configuration is necessary and how the configuration might
> be done.
> 
> # RDF Graphs #
> 
> Let's take as an example:
> 
>
>  dc:title "RDF/XML Syntax Specification (Revised)" ;
>  ex:editor [
>ex:fullName "Dave Beckett" ;
>ex:homePage  ;
>  ] .
> 
> In JSON, I think we'd like to create something like:
> 
>{
>  "$": "http://www.w3.org/TR/rdf-syntax-grammar";,
>  "title": "RDF/XML Syntax Specification (Revised)",
>  "editor": {
>"name": "Dave Beckett",
>"homepage": "http://purl.org/net/dajobe/";
>  }
>}
> 
> Note that the "$" is taken from RDFj. I'm not convinced it's a good
> idea to use this symbol, rather than simply a property called "about"
> or "this" -- any opinions?
> 
> Also note that I've made no distinction in the above between a URI and
> a literal, while RDFj uses <>s around literals. My feeling is that
> normal developers really don't care about the distinction between a
> URI literal and a pointer to a resource, and that they will base the
> treatment of the value of a property on the (name of) the property
> itself.
> 
> So, the first piece of configuration that I think we need here is to
> map properties on to short names that make good JSON identifiers (ie
> name tokens without hyphens). Given that properties normally have
> lowercaseCamelCase local names, it should be possible to use that as a
> default. If you need something more readable, though, it seems like it
> should be possible to use a property of the property, such as:
> 
>ex:fullName api:jsonName "name" .
>ex:homePage api:jsonName "homepage" .
> 
> However, in any particular graph, there may be properties that have
> been given the same JSON name (or, even more probably, local name). We
> could provide multiple alternative names that could be chosen between,
> but any mapping to JSON is going to need to give consistent results
> across a given dataset for people to rely on it as an API, and that
> means the mapping can't be based on what's present in the data. We
> could do something with prefixes, but I have a strong aversion to
> assuming global prefixes.
> 
> So I think this means that we need to provide configuration at an API
> level rather than at a global level: something that can be used
> consistently across a particular API to determine the token that's
> used for a given property. For example:
> 
><> a api:JSON ;
>  api:mapping [
>api:property ex:fullName ;
>api:name "name" ;
>  ] , [
>api:property ex:homePage ;
>api:name "homepage" ;
>  ] .
> 
> There are four more areas where I think there's configuration we need
> to think about:
> 
>* multi-valued properties
>* typed and language-specific values
>* nesting objects
>* suppressing properties
> 
> ## Multi-valued Properties ##
> 
> First one first. It seems obvious that if you have a property with
> multiple values, it should turn into a JSON array structure.

Re: Creating JSON from RDF

[Ivan Herman]

Dan is right on many front... (it usually true:-)

- I personally do not see a problem this being done on ESW, this does not mean 
any type
of formal 'Association' with W3C (giving you a writing right is a matter of 
setting up a
user for you and sending, eg, me or dan a mail to add you to the accepted 
editors' list)

- Indeed, I am experimenting with a MediaWiki for SW Standards. We still have 
to decide
what the 'policy' should be on that, ie, whether it is fine to give pages for 
such
projects. Personally I think it is o.k. but it is not entirely my call, as you 
can
imagine. I will certainly try to clear this up but, with vacations looming and 
I myself
on a trip right now, it may not be clear before 2010.

In both cases these are only wiki pages. Google's offer is larger with a 
mailing list
and such. Yes, W3C is also looking at Mercurial and such, but that is further 
down the
line than the wiki.

Thanks

Ivan



On Sun, December 13, 2009 9:26 pm, Dan Brickley wrote:
> On Sun, Dec 13, 2009 at 8:03 PM, Dave Reynolds
>  wrote:
>> Hi Jeni,
>>
>> [Rest of post snipped for now, I'll respond properly later. Seems like we
>> are on sufficiently similar wavelengths that it is "just" a matter of
>> working the details.]
>>
>>> I don't know where the best place is to work on this: I guess at some
>>> point it would be good to set up a Wiki page or something that we could use
>>> as a hub for discussion?
>>
>> I'd suggest setting up a Google Code area and making anyone who is
>> interested a committer. That gives us a Wiki but also hosting for associated
>> code for generating/navigating the format. I'd be happy to set one up.
>>
>> An alternative is the ESW Wiki but (a) that doesn't have an associated code
>> area, (b) I don't personally have access right now (though I believe that is
>> easily fixable) and (c) it might be presumptuous to associate it with W3C at
>> this stage of baking.
>
> Ivan Herman (cc:'d) has been looking into a modernised general
> 'Semantic Web' wiki area on w3.org, ie. using (Semantic?) MediaWiki,
> rather than the old MoinMoin (for now and forseeable ESW will remain
> using MoinMoin, since migration is non-trivial). There was also some
> recent discussion at W3C about opening up Git or Mercurial distributed
> versioning systems for the standards community, which sounds like it
> could be a good fit for SemWeb IG-and-nearby collaborations. However
> that is at an early stage. Google Code might be easiest for now...
>
> Ivan - care to comment?
>
> Dan
>


-- 
Ivan Herman, W3C Semantic Web Activity Lead
URL: http://www.w3.org/People/Ivan/
FOAF: http://www.ivan-herman.net/foaf.rdf

Re: Creating JSON from RDF

[Toby Inkster]

On Sat, 2009-12-12 at 21:42 +, Jeni Tennison wrote:
> To put this in context, what I think we should aim for is a pure  
> publishing format that is optimised for approachability for normal  
> developers, *not* an interchange format. RDF/JSON [1] and the SPARQL  
> results JSON format [2] aren't entirely satisfactory as far as I'm  
> concerned because of the way the objects of statements are represented
> as JSON objects rather than as simple values. 

jsonGRDDL  provides a method
of mapping from more human-friendly JSON formats to RDF/JSON. (Much like
GRDDL does mapping XML to RDF/XML.) Essentially you want the inverse.

It seems to me that for a format to be very human-friendly, it helps for
it to be fairly domain-specific. For example, if you know you're
modelling people, then it makes sense to have the "sex" property as a
single value, but telephone number, e-mail address, and perhaps even
name might be arrays.

With a deep enough OWL ontology (so that it can be figured out which
properties are singular and plural), I suppose you could get software to
do a reasonably good job creating these human-friendly formats from RDF
data.

-- 
Toby A Inkster

Re: Creating JSON from RDF

[Dan Brickley]

On Sun, Dec 13, 2009 at 8:03 PM, Dave Reynolds
 wrote:
> Hi Jeni,
>
> [Rest of post snipped for now, I'll respond properly later. Seems like we
> are on sufficiently similar wavelengths that it is "just" a matter of
> working the details.]
>
>> I don't know where the best place is to work on this: I guess at some
>> point it would be good to set up a Wiki page or something that we could use
>> as a hub for discussion?
>
> I'd suggest setting up a Google Code area and making anyone who is
> interested a committer. That gives us a Wiki but also hosting for associated
> code for generating/navigating the format. I'd be happy to set one up.
>
> An alternative is the ESW Wiki but (a) that doesn't have an associated code
> area, (b) I don't personally have access right now (though I believe that is
> easily fixable) and (c) it might be presumptuous to associate it with W3C at
> this stage of baking.

Ivan Herman (cc:'d) has been looking into a modernised general
'Semantic Web' wiki area on w3.org, ie. using (Semantic?) MediaWiki,
rather than the old MoinMoin (for now and forseeable ESW will remain
using MoinMoin, since migration is non-trivial). There was also some
recent discussion at W3C about opening up Git or Mercurial distributed
versioning systems for the standards community, which sounds like it
could be a good fit for SemWeb IG-and-nearby collaborations. However
that is at an early stage. Google Code might be easiest for now...

Ivan - care to comment?

Dan

Re: Creating JSON from RDF

[Mark Birbeck]

Hi Jeni,

On Sat, Dec 12, 2009 at 9:42 PM, Jeni Tennison  wrote:
> Hi,
>
> As part of the linked data work the UK government is doing, we're looking at
> how to use the linked data that we have as the basis of APIs that are
> readily usable by developers who really don't want to learn about RDF or
> SPARQL.

Great.


> One thing that we want to do is provide JSON representations of both RDF
> graphs and SPARQL results. I wanted to run some ideas past this group as to
> how we might do that.

Great again. :)

In the work I've been doing, I've concluded that in JSON-world, an RDF
graph should be a JSON object (as explained in RDFj [1], and as you
seem to concur), but also that SPARQL queries should return RDFj
objects too.

In other words, after a lot of playing around I concluded that there
was nothing to be gained from differentiating between representations
of graphs, and the results of queries.


> To put this in context, what I think we should aim for is a pure publishing
> format that is optimised for approachability for normal developers, *not* an
> interchange format. RDF/JSON and the SPARQL results JSON format
> aren't entirely satisfactory as far as I'm concerned because of the way the
> objects of statements are represented as JSON objects rather than as simple
> values. I still think we should produce them (to wean people on to, and for
> those using more generic tools), but I'd like to think about producing
> something that is a bit more immediately approachable too.

+72. :)

I would also put irJSON into this category, which I see is referred to
in a later post.


> RDFj is closer to what I think is needed here.

Good.


> However, I don't think
> there's a need for setting 'context' given I'm not aiming for an interchange
> format, there are no clear rules about how to generate it from an arbitrary
> graph (basically there can't be without some additional configuration) and
> it's not clear how to deal with datatypes or languages.

I probably didn't explain the use of 'context' well enough, but since
I think you do need it, I'll explain it more, below.


> I suppose my first question is whether there are any other JSON-based
> formats that we should be aware of, that we could use or borrow ideas from?

I also did a thorough search, before devising RDFj.

In general I found many 'interchange formats', as you call them, but I
didn't find anything that came from the other direction, saying 'how
should we interpret JavaScript data as RDF'.

I think this approach is what makes RDFj different, because it is
trying as much as possible to leave the JS alone, and provide a layer
of /interpretation/. It's exactly how I approached RDFa -- I began
with HTML mark-up, such as ,  and , and asked myself
what an RDF interpretation of each 'pattern' would be.

(Which incidentally shows that there is plenty more work that could be
done on this, in RDFa; what is an RDF interpretation of @cite,
, and , for example?)


> Assuming there aren't, I wanted to discuss what generic rules we might use,
> where configuration is necessary and how the configuration might be done.

Excellent. It may sound sacrilegious, but I happen to think that in
lots of ways RDFj is more important than RDFa. Consequently, I was
beginning to worry that everyone was quite happy with the 'interchange
formats', and didn't see the point of discussing a more 'natural' JSON
approach!


> # RDF Graphs #
>
> Let's take as an example:
>
>  
>    dc:title "RDF/XML Syntax Specification (Revised)" ;
>    ex:editor [
>      ex:fullName "Dave Beckett" ;
>      ex:homePage  ;
>    ] .
>
> In JSON, I think we'd like to create something like:
>
>  {
>    "$": "http://www.w3.org/TR/rdf-syntax-grammar";,
>    "title": "RDF/XML Syntax Specification (Revised)",
>    "editor": {
>      "name": "Dave Beckett",
>      "homepage": "http://purl.org/net/dajobe/";
>    }
>  }

Definitely.

Key things are that -- other than the subject -- it's very familiar to
JS programmers. In particular, there's no verbose use of 'name' and
'value' properties to demarcate the predicates and objects, in the way
that 'interchange formats' do. Also, there are no explicit bnodes to
indicate that one statement's subject is another's object -- the
natural flow of JavaScript is used.

These were key design goals for RDFj.


> Note that the "$" is taken from RDFj. I'm not convinced it's a good idea to
> use this symbol, rather than simply a property called "about" or "this" --
> any opinions?

I agree, and in my RDFj description I do say that since '$' is used in
a lot of Ajax libraries, I should find something else.

However, in my view, the 'something else' shouldn't look like a
predicate, so I don't think 'about' or 'this' (or 'id' as someone
suggests later in the thread), should be used. (Note also that 'id' is
used in a related but slightly different way by Dojo.)

Also, the underscore is generally related to bnodes,

Re: Creating JSON from RDF

[Dave Reynolds]

Hi Jeni,

[Rest of post snipped for now, I'll respond properly later. Seems like 
we are on sufficiently similar wavelengths that it is "just" a matter of 
working the details.]


I don't know where the best place is to work on this: I guess at some 
point it would be good to set up a Wiki page or something that we could 
use as a hub for discussion?


I'd suggest setting up a Google Code area and making anyone who is 
interested a committer. That gives us a Wiki but also hosting for 
associated code for generating/navigating the format. I'd be happy to 
set one up.


An alternative is the ESW Wiki but (a) that doesn't have an associated 
code area, (b) I don't personally have access right now (though I 
believe that is easily fixable) and (c) it might be presumptuous to 
associate it with W3C at this stage of baking.


Cheers,
Dave

Re: Creating JSON from RDF

[Jeni Tennison]

Hi Dave :)

On 13 Dec 2009, at 13:34, Dave Reynolds wrote:

Jeni Tennison wrote:
As part of the linked data work the UK government is doing, we're  
looking at how to use the linked data that we have as the basis of  
APIs that are readily usable by developers who really don't want to  
learn about RDF or SPARQL.


Wow! Talk about timing. We are looking at exactly the same issue as  
part of the TSB work and were starting to look at JSON formats just  
this last couple of days. We should combine forces.


Excellent :)

One thing that we want to do is provide JSON representations of  
both RDF graphs and SPARQL results. I wanted to run some ideas past  
this group as to how we might do that.


I agree we want both graphs and SPARQL results but I think there is  
another third case - lists of described objects.


I absolutely agree with you that lists of described objects is an  
essential part of an API. In fact, I was going to (and will!) write a  
separate message about possible approaches for creating such lists.


It seemed to me that lists could be represented with RDF like:

  
rdfs:label "Local Authorities - Page 1" ;
xhv:next  ;

...
api:contents (
  
  
  
  ...
)

This is just RDF, and as such any rules that we create about mapping  
RDF graphs to JSON could apply. (I agree that the list page should  
include extra information about the items in the list, but that seems  
to me to be a separable issue.)


[snip]
RDFj [3] is closer to what I think is needed here. However, I don't  
think there's a need for setting 'context' given I'm not aiming for  
an interchange format, there are no clear rules about how to  
generate it from an arbitrary graph (basically there can't be  
without some additional configuration) and it's not clear how to  
deal with datatypes or languages.


WRT 'context' you might not need it but it I don't think it is  
harmful.  I think if we said to developers that there is some outer  
wrapper like:


{
  "format" : "RDF-JSON",
  "version" : "0.1",
  "mapping" :  ... magic stuff ...
  "data" : ... the bit you care about ...
}

The developers would be quite happy doing that one dereference and  
ignore the mapping stuff but it might allow inversion back to RDF  
for those few who do care, or come to care.


OK. I agree that the 'come to care' is important. What I don't want is  
for concerns about round-tripping JSON data to override concerns about  
the usability of the resulting format.


I suppose my first question is whether there are any other JSON- 
based formats that we should be aware of, that we could use or  
borrow ideas from?


The one that most intrigued me as a possible starting point was the  
Simile Exhibit JSON format [1]. It is developer friendly in much the  
way that you talk about but it has the advantage of zero  
configuration, some measure of invertability, has an online  
translator [2] and is supported by the RPI Sparql proxy [3].


I've some reservations about standardizing on it as is:
- lack of documentation of the mapping
- some inconsistencies in how references between resources are  
encoded (at least judging by the output of Babel[2] on test cases)
- handling of bNodes - I'd rather single referenced bNodes were  
serialized as nested structures


I agree, that looks pretty good but needs a bit of work.

One thing it makes me think is that perhaps JSON Schema [1] could form  
the basis of the mechanism for expressing any extra stuff that's  
required about the properties.


One starting assumption to call out: I'd like to aim for a zero  
configuration option and that explicit configuration is only used to  
help tidy things up but isn't required to get started.


Agreed.

In terms of details I was thinking of following the Simile  
convention on short form naming that, in the absence of clashes, use  
the rdfs:label falling back to the localname, as the basis for the  
shortened property names. So knowing nothing else the bNode would be:


 ...
   "editor": {
  "fullName": "Dave Beckett",
  "homePage": "http://purl.org/net/dajobe/";
   }

In the event of clashes then fall back on a prefix based  
disambiguation.


Agreed. I rather deliberately and unnecessarily chose to change the  
names for the example in order to demonstrate the principle.


Note that the "$" is taken from RDFj. I'm not convinced it's a good  
idea to use this symbol, rather than simply a property called  
"about" or "this" -- any opinions?


I'd prefer "id" (though "about" is OK), "$" is too heavily overused  
in javascript libraries.


I agree. From the brief survey of JSON APIs that I did just now, it  
seems as though prefixing a reserved property n

Re: Creating JSON from RDF

[Jeni Tennison]

On 12 Dec 2009, at 22:33, Damian Steer wrote:
In terms of code accessing the results there isn't much difference.  
The former requires an initial path to the array ['results'] 
['bindings'], then the use of ['value'] for each access.



Is true. As I said, I'm a lot less concerned about the JSON from  
SPARQL results than from arbitrary RDF graphs. Let's focus on that.


Jeni
--
Jeni Tennison
http://www.jenitennison.com

Re: Creating JSON from RDF

[Jeni Tennison]

On 12 Dec 2009, at 22:27, Danny Ayers wrote:

I can't offer any real practical suggestions right away (a lot to
digest here!) but one question I think right away may some
significance: you want this to be friendly to normal developers - what
kind of things are they actually used to? Do you have any examples of
existing JSON serializations of relatively complex data structures -
something to give an idea of the target.

While it's a bit apples and oranges, there presumably are plenty of
systems now pushing out JSON rather than the XML they would a few
years ago - is there anything to be learnt from that scenario that can
be exploited for RDF?


Great idea to look at what currently exists. Let's see.

The flickr API [1] is notable in that the JSON is a direct mapping  
from the XML API. From what I can tell they don't even try to use  
values that are anything other than strings, and they have a  
moderately ugly "_content" property for when elements in the XML API  
have content (though it mostly uses attributes, from what I can tell).


The Twitter API [2] provides JSON and XML (Atom) formats. There are a  
whole bunch of different queries you can do, most of which contain  
nested objects etc. Interestingly, the JSON that you get back from a  
search does contain metadata about the search and a 'results' property  
that holds an array of the results. So that could be a relevant model  
for SPARQL results formats.


CouchDB [3] is a purely JSON API which has to be very generic (since  
CouchDB can be used to contain anything). It uses reserved property  
names like "_id" (this and "_content" in the flickr API make me think  
that a leading underscore is the expected way to create reserved  
property names).


Yahoo! [4] have a JSON API that is again based on an XML API with a  
straight-forward mapping from the XML to the JSON.


The Google Data Protocol [5] uses JSON that is generated from the  
equivalent Atom feed. Interestingly, they provide support for  
namespaces by having "xmlns" properties (with "$" used instead of ":"  
in any qualified names). Unlike the other APIs, they do use  
namespaces, but only a handful. I strongly doubt that any developer  
using it would actually resolve "openSearch$startIndex" by looking at  
the "xmlns$openSearch" property.


Is that enough of an idea to be getting on with?

It's worth noting that most of these APIs support a callback=  
parameter that makes the API return Javascript containing a function  
call rather than simply the JSON itself. I regard this as an  
unquestionably essential part of a JSON API, whether it uses RDF/JSON  
or RDFj or irJSON or whatever, in order to make the JSON usable cross- 
site.


Cheers,

Jeni

[1]: http://www.flickr.com/services/api/response.json.html
[2]: http://apiwiki.twitter.com/Twitter-API-Documentation
[3]: http://wiki.apache.org/couchdb/HTTP_Document_API
[4]: http://developer.yahoo.com/common/json.html
[5]: http://code.google.com/apis/gdata/docs/json.html
--
Jeni Tennison
http://www.jenitennison.com

Re: Creating JSON from RDF

[Dave Reynolds]

Hi Jeni,

Jeni Tennison wrote:

As part of the linked data work the UK government is doing, we're 
looking at how to use the linked data that we have as the basis of APIs 
that are readily usable by developers who really don't want to learn 
about RDF or SPARQL.


Wow! Talk about timing. We are looking at exactly the same issue as part 
of the TSB work and were starting to look at JSON formats just this last 
couple of days. We should combine forces.


One thing that we want to do is provide JSON representations of both RDF 
graphs and SPARQL results. I wanted to run some ideas past this group as 
to how we might do that.


I agree we want both graphs and SPARQL results but I think there is 
another third case - lists of described objects.


This seems to have been a common pattern in the apps that I've worked 
on. You want to find all objects (resources in RDF speak) that match 
some criteria, with some ordering, and get back a list of them and their 
associated properties. This is like a SPARQL DESCRIBE operating on each 
of an ordered list of resources found by a SPARQL SELECT.


The point is that this is not a graph because the top level list needs 
to be ordered. It is not a SPARQL result set because you want the 
descriptions to include any of the properties that are present in the 
data (potentially included bNode closure) without having to know all 
those and spell them out in the query. But it is a natural thing to want 
to return from a REST API.


To put this in context, what I think we should aim for is a pure 
publishing format that is optimised for approachability for normal 
developers, *not* an interchange format. RDF/JSON [1] and the SPARQL 
results JSON format [2] aren't entirely satisfactory as far as I'm 
concerned because of the way the objects of statements are represented 
as JSON objects rather than as simple values. I still think we should 
produce them (to wean people on to, and for those using more generic 
tools), but I'd like to think about producing something that is a bit 
more immediately approachable too.


RDFj [3] is closer to what I think is needed here. However, I don't 
think there's a need for setting 'context' given I'm not aiming for an 
interchange format, there are no clear rules about how to generate it 
from an arbitrary graph (basically there can't be without some 
additional configuration) and it's not clear how to deal with datatypes 
or languages.


WRT 'context' you might not need it but it I don't think it is harmful. 
 I think if we said to developers that there is some outer wrapper like:


{
   "format" : "RDF-JSON",
   "version" : "0.1",
   "mapping" :  ... magic stuff ...
   "data" : ... the bit you care about ...
}

The developers would be quite happy doing that one dereference and 
ignore the mapping stuff but it might allow inversion back to RDF for 
those few who do care, or come to care.


I suppose my first question is whether there are any other JSON-based 
formats that we should be aware of, that we could use or borrow ideas from?


The one that most intrigued me as a possible starting point was the 
Simile Exhibit JSON format [1]. It is developer friendly in much the way 
that you talk about but it has the advantage of zero configuration, some 
measure of invertability, has an online translator [2] and is supported 
by the RPI Sparql proxy [3].


I've some reservations about standardizing on it as is:
 - lack of documentation of the mapping
 - some inconsistencies in how references between resources are encoded 
(at least judging by the output of Babel[2] on test cases)
 - handling of bNodes - I'd rather single referenced bNodes were 
serialized as nested structures


[There was another format we used in a project in my previous existence 
but I'm not sure if that was made public anywhere, will check.]


Assuming there aren't, I wanted to discuss what generic rules we might 
use, where configuration is necessary and how the configuration might be 
done.


One starting assumption to call out: I'd like to aim for a zero 
configuration option and that explicit configuration is only used to 
help tidy things up but isn't required to get started.



# RDF Graphs #

Let's take as an example:

  
dc:title "RDF/XML Syntax Specification (Revised)" ;
ex:editor [
  ex:fullName "Dave Beckett" ;
  ex:homePage  ;
] .

In JSON, I think we'd like to create something like:

  {
"$": "http://www.w3.org/TR/rdf-syntax-grammar";,
"title": "RDF/XML Syntax Specification (Revised)",
"editor": {
  "name": "Dave Beckett",
  "homepage": "http://purl.org/net/dajobe/";
}
  }


+1 on style

In terms of details I was thinking of following the Simile convention on 
short form naming that, in the absence of clashes, use the rdfs:label 
falling back to the localname, as the basis for the shortened property 
names. So knowing nothing else the bNode would be:


  .

Re: Creating JSON from RDF

[Jeni Tennison]

Frederick,

Thanks, that looks interesting, and the idea of supporting similar  
mappings to XML and to CSV is very attractive.


But I couldn't actually work out how I would use it in the kind of  
situation we find ourselves in. We have, for example, RDF like this:


  http://education.data.gov.uk/doc/school/520965.rdf

which we want to make readily usable by people creating mash-ups. What  
are the (minimal) definitions that we have to create to use irON to  
create customised JSON/CSV/XML from this RDF?


Thanks,

Jeni

On 12 Dec 2009, at 22:00, Frederick Giasson wrote:


Hi Jeni,

One project, amongst other, in this trend is irON [1] (with its  
irJSON[2] serialization profile).


Also note that a revision of this specification will be released in  
the coming month or so based on the latest work we have done  
regarding the development of some parsers. The biggest areas that  
has been enhanced is  related to the "schema".


I think this project is in direct relation with the goal you stated:  
"what I think we should aim for is a pure publishing format that is  
optimised for approachability for normal developers, *not* an  
interchange format."


Another thing I really find interesting with irON is its commON[3] 
[4] profile (CSV) which has been developed to easily create and  
manage datasets of records description in spreadsheet applications  
(mainly Excel and OpenOffice Calculator).


Do not hesitate to ask any questions regarding irON or any of its  
profile, questions about some design decisions we take that lead to  
this specification, etc.




[1]  http://openstructs.org/iron/iron-specification
[2]  http://openstructs.org/iron/iron-specification#mozTocId462570
[3]  http://openstructs.org/iron/iron-specification#mozTocId603499
[4]  http://www.mkbergman.com/845/a-most-un-common-way-to-author-datasets/



Thanks!


Take care,


Fred



--
Jeni Tennison
http://www.jenitennison.com

Re: Creating JSON from RDF

[Nathan]

Damian Steer wrote:
> On 12 Dec 2009, at 21:42, Jeni Tennison wrote:
> 
>> Hi,
>>
>> As part of the linked data work the UK government is doing, we're looking at 
>> how to use the linked data that we have as the basis of APIs that are 
>> readily usable by developers who really don't want to learn about RDF or 
>> SPARQL.
> 
>> # SPARQL Results #
> 
> 
>>  {
>>"head": {
>>  "vars": [ "book", "title" ]
>>},
>>"results": {
>>  "bindings": [
>>{
>>  "book": {
>>"type": "uri",
>>"value": "http://example.org/book/book6";
>>  },
>>  "title": {
>>"type": "literal",
>>"value", "Harry Potter and the Half-Blood Prince"
>>  }
>>},
>>{
>>  "book": {
>>"type": "uri",
>>"value": "http://example.org/book/book5";
>>  },
>>  "title": {
>>"type": "literal",
>>"value": "Harry Potter and the Order of the Phoenix"
>>  }
>>},
>>...
>>  ]
>>}
>>  }
>>
>> a normal developer would want to just get:
>>
>>  [{
>>"book": "http://example.org/book/book6";,
>>"title": "Harry Potter and the Half-Blood Prince"
>>   },{
>> "book": "http://example.org/book/book5";,
>> "title": "Harry Potter and the Order of the Phoenix"
>>   },
>>   ...
>>  ]
> 

with regards sparql query results in json - these can be handled no
problem on the client side (10 minutes passes) to illustrate here's a
PHP 5 class to do exactly as your example above specified:

class SparqlResultsHandler
{
  private static $__instance;

  const TYPE_URI = 'uri';
  const TYPE_BNODE = 'bnode';
  const TYPE_LITERAL = 'literal';
  const DATATYPE_XML =
'http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral';
  const DATATYPE_DATE = 'http://www.w3.org/2001/XMLSchema#dateTime';

  const DATATYPE_STRING = 'http://www.w3.org/2001/XMLSchema#string';
  const DATATYPE_FLOAT = 'http://www.w3.org/2001/XMLSchema#float';
  const DATATYPE_BOOL = 'http://www.w3.org/2001/XMLSchema#boolean';
  const DATATYPE_INT = 'http://www.w3.org/2001/XMLSchema#int';


  public static function getValue( $value )
  {
if( !is_object($value) || !isset($value->value) ) return $value;
if( $value->type == 'bnode' ) return null;
if( !isset($value->datatype) ) return $value->value;
switch( $value->datatype ) {
  case self::DATATYPE_BOOL:
return (bool)$value->value;
break;
  case self::DATATYPE_DATE:
return strtotime($value->value);
break;
  case self::DATATYPE_FLOAT:
return (float)$value->value;
break;
  case self::DATATYPE_INT:
return (int)$value->value;
break;
  case self::DATATYPE_STRING:
return (string)$value->value;
break;
  case self::DATATYPE_XML:
$value = (string)$value->value;
$value = str_replace( '
xmlns:xhtml="http://www.w3.org/1999/xhtml";' , '' , $value );
$value = str_replace( 'xhtml:' , '' , $value );
$value = str_replace( '' , '' , $value );
$value = str_replace( '>' , ' />' , $value );
return $value;
break;
  default:
return $value->value;
break;
}
  }

  public static function getBindings( $sparql_results )
  {
if( !is_object($sparql_results) ) {
  $sparql_results = json_decode( $sparql_results );
}
return $sparql_results->results->bindings;
  }

  public static function handleResults( $sparql_results )
  {
$results = self::getBindings( $sparql_results );
$tiny = array();
foreach( $results as $result ) {
  $tempResult = new stdClass();
  foreach( $result as $name => $param ) {
$tempResult->{$name} = self::getValue($param);
  }
  $tiny[] = $tempResult;
}
return $tiny;
  }

}

$json = '{ "head": { "link": [], "vars": ["counter"] },
  "results": { "distinct": false, "ordered": true, "bindings": [
{ "counter": { "type": "typed-literal", "datatype":
"http://www.w3.org/2001/XMLSchema#integer";, "value": "3021" }} ] } }';

print_r( SparqlResultsHandler::handleResults($json) );


output of the above:

Array
(
[0] => stdClass Object
(
[counter] => 3021
)

)


... back to text ...

I guess what I'm trying to say is, I'm a normal developer and handling
sparql results really isn't a problem; RDF is a different matter that
can also be handled with just a little bit of client side code and the
use of a lib or two, but that's a different matter.

many regards,

nathan

Re: Creating JSON from RDF

[Frederick Giasson]

Hi Kingsley,
Why not use Jeni's dump as a usecase example/tutorial re. irON? A What 
(for the problem), Why (for irON virtues) and a How (an example based 
on the usecase presented). This is how you can formulate a very sharp 
apex for the irON value pyramid. Also note, UK, US, and Aussie 
governments (amongst others) are all hitting the publishing of 
structured data hump.


The specification tries to answer the What, Why and How questions. 
Additionally, Mike did write some blog posts that further explains the 
What, Why and How of irON [1] [2]. We also released a case study of 
commON with the Semantic Web Sweet Tools list [3].


Finally some more usecases (irJSON ones amongst other) will be released 
in the future based on current work for clients.




[1]  http://www.mkbergman.com/838/iron-semantic-web-for-mere-mortals/
[2]  http://www.mkbergman.com/845/a-most-un-common-way-to-author-datasets/
[3]  http://openstructs.org/iron/common-swt-annex



Thanks!


Take care,


Fred

Re: Creating JSON from RDF

[Nathan]

Jeni Tennison wrote:
> Hi,
> 
> As part of the linked data work the UK government is doing, we're
> looking at how to use the linked data that we have as the basis of APIs
> that are readily usable by developers who really don't want to learn
> about RDF or SPARQL.
> 
> One thing that we want to do is provide JSON representations of both RDF
> graphs and SPARQL results. I wanted to run some ideas past this group as
> to how we might do that.
> 

[big snip]

in all honesty, I strongly feel this should be handled on the
clientside, keep everything in rdf is good and serves all audiences, for
those who don't want to learn it, then why not provide client side
libraries to consume the rdf and turn it in to a format more preferable
for them?

Personally I just created some helper classes (details are out of the
scope of this) but essentially they let me use the following syntax
against RDF

print object->{'foaf:name'};

which I hope you'll agree really isn't any more complex than:

print object->name

Additionally it gently introduces developers to the notion of RDF,
predicates and that there really is a better way to be doing things; but
keeps it as a learning curve they can choose to take at their own time.

And finally the above approach of keeping the predicate but using
namespace prefixes will work for all rdf, not just a small subset of rdf
graphs, thus saving the "normal developer" from learning 100's of
different API's.

Many Regards,

Nathan

Re: Creating JSON from RDF

[Nathan]

Damian Steer wrote:
> On 12 Dec 2009, at 21:42, Jeni Tennison wrote:
> 
>> Hi,
>>
>> As part of the linked data work the UK government is doing, we're looking at 
>> how to use the linked data that we have as the basis of APIs that are 
>> readily usable by developers who really don't want to learn about RDF or 
>> SPARQL.
> 
>> # SPARQL Results #
> 
> 
>>  {
>>"head": {
>>  "vars": [ "book", "title" ]
>>},
>>"results": {
>>  "bindings": [
>>{
>>  "book": {
>>"type": "uri",
>>"value": "http://example.org/book/book6";
>>  },
>>  "title": {
>>"type": "literal",
>>"value", "Harry Potter and the Half-Blood Prince"
>>  }
>>},
>>{
>>  "book": {
>>"type": "uri",
>>"value": "http://example.org/book/book5";
>>  },
>>  "title": {
>>"type": "literal",
>>"value": "Harry Potter and the Order of the Phoenix"
>>  }
>>},
>>...
>>  ]
>>}
>>  }
>>
>> a normal developer would want to just get:
>>
>>  [{
>>"book": "http://example.org/book/book6";,
>>"title": "Harry Potter and the Half-Blood Prince"
>>   },{
>> "book": "http://example.org/book/book5";,
>> "title": "Harry Potter and the Order of the Phoenix"
>>   },
>>   ...
>>  ]
> 
> In terms of code accessing the results there isn't much difference. The 
> former requires an initial path to the array ['results']['bindings'], then 
> the use of ['value'] for each access.
> 
> Damian
> 

apologies if this duplicates, send from wrong address previously!

with regards sparql query results in json - these can be handled no
problem on the client side (10 minutes passes) to illustrate here's a
PHP 5 class to do exactly as your example above specified:

class SparqlResultsHandler
{
  private static $__instance;

  const TYPE_URI = 'uri';
  const TYPE_BNODE = 'bnode';
  const TYPE_LITERAL = 'literal';
  const DATATYPE_XML =
'http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral';
  const DATATYPE_DATE = 'http://www.w3.org/2001/XMLSchema#dateTime';

  const DATATYPE_STRING = 'http://www.w3.org/2001/XMLSchema#string';
  const DATATYPE_FLOAT = 'http://www.w3.org/2001/XMLSchema#float';
  const DATATYPE_BOOL = 'http://www.w3.org/2001/XMLSchema#boolean';
  const DATATYPE_INT = 'http://www.w3.org/2001/XMLSchema#int';


  public static function getValue( $value )
  {
if( !is_object($value) || !isset($value->value) ) return $value;
if( $value->type == 'bnode' ) return null;
if( !isset($value->datatype) ) return $value->value;
switch( $value->datatype ) {
  case self::DATATYPE_BOOL:
return (bool)$value->value;
break;
  case self::DATATYPE_DATE:
return strtotime($value->value);
break;
  case self::DATATYPE_FLOAT:
return (float)$value->value;
break;
  case self::DATATYPE_INT:
return (int)$value->value;
break;
  case self::DATATYPE_STRING:
return (string)$value->value;
break;
  case self::DATATYPE_XML:
$value = (string)$value->value;
$value = str_replace(
'xmlns:xhtml="http://www.w3.org/1999/xhtml";' , '' , $value );
$value = str_replace( 'xhtml:' , '' , $value );
$value = str_replace( '' , '' , $value );
$value = str_replace( '>' , ' />' , $value );
return $value;
break;
  default:
return $value->value;
break;
}
  }

  public static function getBindings( $sparql_results )
  {
if( !is_object($sparql_results) ) {
  $sparql_results = json_decode( $sparql_results );
}
return $sparql_results->results->bindings;
  }

  public static function handleResults( $sparql_results )
  {
$results = self::getBindings( $sparql_results );
$tiny = array();
foreach( $results as $result ) {
  $tempResult = new stdClass();
  foreach( $result as $name => $param ) {
$tempResult->{$name} = self::getValue($param);
  }
  $tiny[] = $tempResult;
}
return $tiny;
  }

}

$json = '{ "head": { "link": [], "vars": ["counter"] },
  "results": { "distinct": false, "ordered": true, "bindings": [
{ "counter": { "type": "typed-literal", "datatype":
"http://www.w3.org/2001/XMLSchema#integer";, "value": "3021" }} ] } }';

print_r( SparqlResultsHandler::handleResults($json) );


output of the above:

Array
(
[0] => stdClass Object
(
[counter] => 3021
)

)


.. back to text ...

I guess what I'm trying to say is, I'm a normal developer and handling
sparql results really isn't a problem; RDF is a different matter that
can also be handled with just a little bit of client side code and the
use of a lib or two, but that's a different matter.

many regards,

nathan

Re: Creating JSON from RDF

[Damian Steer]

On 12 Dec 2009, at 21:42, Jeni Tennison wrote:

> Hi,
> 
> As part of the linked data work the UK government is doing, we're looking at 
> how to use the linked data that we have as the basis of APIs that are readily 
> usable by developers who really don't want to learn about RDF or SPARQL.

> # SPARQL Results #


> 
>  {
>"head": {
>  "vars": [ "book", "title" ]
>},
>"results": {
>  "bindings": [
>{
>  "book": {
>"type": "uri",
>"value": "http://example.org/book/book6";
>  },
>  "title": {
>"type": "literal",
>"value", "Harry Potter and the Half-Blood Prince"
>  }
>},
>{
>  "book": {
>"type": "uri",
>"value": "http://example.org/book/book5";
>  },
>  "title": {
>"type": "literal",
>"value": "Harry Potter and the Order of the Phoenix"
>  }
>},
>...
>  ]
>}
>  }
> 
> a normal developer would want to just get:
> 
>  [{
>"book": "http://example.org/book/book6";,
>"title": "Harry Potter and the Half-Blood Prince"
>   },{
> "book": "http://example.org/book/book5";,
> "title": "Harry Potter and the Order of the Phoenix"
>   },
>   ...
>  ]

In terms of code accessing the results there isn't much difference. The former 
requires an initial path to the array ['results']['bindings'], then the use of 
['value'] for each access.

Damian

Re: Creating JSON from RDF

[Danny Ayers]

Jeni, marvellous that you're working on this, and marvellous that
you've thought this through an awful lot already.

I can't offer any real practical suggestions right away (a lot to
digest here!) but one question I think right away may some
significance: you want this to be friendly to normal developers - what
kind of things are they actually used to? Do you have any examples of
existing JSON serializations of relatively complex data structures -
something to give an idea of the target.

While it's a bit apples and oranges, there presumably are plenty of
systems now pushing out JSON rather than the XML they would a few
years ago - is there anything to be learnt from that scenario that can
be exploited for RDF?

Cheers,
Danny.


-- 
http://danny.ayers.name

Re: Creating JSON from RDF

[Kingsley Idehen]

Frederick Giasson wrote:

Hi Jeni,

One project, amongst other, in this trend is irON [1] (with its 
irJSON[2] serialization profile).


Also note that a revision of this specification will be released in 
the coming month or so based on the latest work we have done regarding 
the development of some parsers. The biggest areas that has been 
enhanced is  related to the "schema".


I think this project is in direct relation with the goal you stated: 
"what I think we should aim for is a pure publishing format that is 
optimised for approachability for normal developers, *not* an 
interchange format."


Another thing I really find interesting with irON is its commON[3][4] 
profile (CSV) which has been developed to easily create and manage 
datasets of records description in spreadsheet applications (mainly 
Excel and OpenOffice Calculator).


Do not hesitate to ask any questions regarding irON or any of its 
profile, questions about some design decisions we take that lead to 
this specification, etc.




[1]  http://openstructs.org/iron/iron-specification
[2]  http://openstructs.org/iron/iron-specification#mozTocId462570
[3]  http://openstructs.org/iron/iron-specification#mozTocId603499
[4]  
http://www.mkbergman.com/845/a-most-un-common-way-to-author-datasets/


Fred,

Why not use Jeni's dump as a usecase example/tutorial re. irON? A What 
(for the problem), Why (for irON virtues) and a How (an example based on 
the usecase presented). This is how you can formulate a very sharp apex 
for the irON value pyramid. Also note, UK, US, and Aussie governments 
(amongst others) are all hitting the publishing of structured data hump.


Kingsley




Thanks!


Take care,


Fred





--


Regards,

Kingsley Idehen   Weblog: http://www.openlinksw.com/blog/~kidehen
President & CEO 
OpenLink Software Web: http://www.openlinksw.com

Re: Creating JSON from RDF

[Frederick Giasson]

Hi Jeni,

One project, amongst other, in this trend is irON [1] (with its 
irJSON[2] serialization profile).


Also note that a revision of this specification will be released in the 
coming month or so based on the latest work we have done regarding the 
development of some parsers. The biggest areas that has been enhanced 
is  related to the "schema".


I think this project is in direct relation with the goal you stated: 
"what I think we should aim for is a pure publishing format that is 
optimised for approachability for normal developers, *not* an 
interchange format."


Another thing I really find interesting with irON is its commON[3][4] 
profile (CSV) which has been developed to easily create and manage 
datasets of records description in spreadsheet applications (mainly 
Excel and OpenOffice Calculator).


Do not hesitate to ask any questions regarding irON or any of its 
profile, questions about some design decisions we take that lead to this 
specification, etc.




[1]  http://openstructs.org/iron/iron-specification
[2]  http://openstructs.org/iron/iron-specification#mozTocId462570
[3]  http://openstructs.org/iron/iron-specification#mozTocId603499
[4]  http://www.mkbergman.com/845/a-most-un-common-way-to-author-datasets/



Thanks!


Take care,


Fred