Chris Beer wrote:
Thanks for the reply Kingsley./Kingsley Idehen wrote:/What I was in fact suggesting here is that we clearly define the difference between Data being Open as in access and policy surrounding it - the political/philosophical side of the coin, and Data being Open as in Standards and, as you so better put it - structured data - the technical side of the coin. The semantics surrounding the two are important - to date we have basically said in e-Gov IG "Lets make Open (Standard) Data Open (to the Public)" - anyone coming in with no background knowledge - potentially such as as those working in policy from a non-IT background that is covered by initial Working Draft <http://www.w3.org/TR/gov-data/>'s "/To: Any government wishing to set-up data.gov.*" /(wiki version), is simply going to start to find it confusing. We have discussions on defining open data that are centered around the access/policy question, and we have discussions on Linked Open Gov Data that are veering into the technical. For the good of all involved, I feel we need to define, set, and stick to some basic terminology that doesn't confuse the two./Chris Beer wrote: //Structured Data based on industry standard data representation formats. Just as UNIX came down to POSIX. Ditto Internet re. TCP/IP. Openness is about Standards, and has nothing to do with politics or philosophy./I think Thomas makes some excellent points. Is it possible as a group to agree on something akin to the following?1) Open Data refers to how data is accessed and is primarily a political/policy consideration/You can institute policies that mandate the use of industry standard data formats re. data placed in the public domain or simply published for reuse by others.//2) Linked (Open) Data refers to how data is structured and delivered and is primarily a technological/standards consideration/To be precise: HTTP based Linked Open Data. This is about the incorporation of HTTP scheme Identifiers into data that has be published using a standard data representation format./Note: to get data into any standard data representation format there has to be a formal data model. At the most basic, said model takes the form: Entity-Attribute-Value. In the case of Linked Open Data, you have the intersection of the following:1. EAV model 2. Standard Data Formats 3. HTTP scheme Identifiers (HTTP URIs)./
Amen!
What we really have to invest some time in re., general communications, is genuine appreciation the fact that HTTP based Linked Data, HTTP based Linked Open Data, or anything else associated with the EAV/CR with HTTP URIs model, comes down to embracing and extending time-tested best practices from the pre-Web DBMS realm. Basically, ignoring the DBMS realm (covertly or overtly) simply puts this whole endeavor in the: "Pot is Broken, so you now own it", bucket.Quite simply - It is Government that conducts the majority of hard statistical research and collates data. DBpedia, or indeed any other commercial enterprise, including Academia, does not equal the sum total of Linked Open Data. They do indeed provide a pivotal and valuable service - but only in the sense that Google does with searches. They are a "reseller" of Data in that sense - but Government is, and will remain for a long time, the primary producer of raw datasets./3) The majority of datasets, LOD or not, that are of real value, are developed, maintained and delivered by Government, like it or not. We know this without even looking at the LOD Projects work <http://esw.w3.org/topic/SweoIG/TaskForces/CommunityProjects/LinkingOpenData#head-277d7f68544ce1a9e252f5c0080b6402cd983a49> (which interestingly, contains very little Government data, which is a worry as it possibly indicates that Governments just AREN'T getting on board with early take up of LOD, despite the various legal requirements coming out world wide)./How have you arrived at the above bearing in mind the pivotal role of DBpedia? Basically, this is about a Linked Open Data Space derived from Wikipedia snapshots which have little or no Govt. data. Of course, things get much better across depth, quality, and linked density dimensions when Govt. data is cross linked with LOD spaces like DBpedia etc.//
Mater-Details associations between tables in a DBMS is one of the most basic data interaction patterns, its typically underlies construction of user oriented VIEWs and data entry forms. This very pattern reemerges in the Linked Data realm, but the roles of Master and Details become relative due to inherent contextual fluidity that pervades HTTP based Linked Data openness. Thus, in the eyes of the Government, publishing raw data in structured form is akin to publishing a Master lookup table; and from this perspective DBpedia, is an associated Details oriented Table. Likewise, In the eyes of DBpedia, the inverse is the case i.e., it see Govt Data (or any other LOD cloud bubble) as an associated Details Table.
As described above, the fluid Master-Details details dynamic across Data Spaces (conceptual equivalent of Tables / Table Spaces in RDBMs realm), ultimately implies that all contributions to a Linked Open Data Cloud are valuable, and the degree of value is as subjective as "beauty" i.e., lies totally in the eyes (context) of the beholder :-)
If there were huge chunks of Government Datasets floating around in the public domain waiting to be linked, it would of been done by someone already, and we wouldn't be having this discussion. As Thomas points out: /"we have tons of government data with a legal obligation to make them available to the public (at least in Europe, and especially environmental data), and we are looking for means to do so in the most efficient way." /While he is referring to the technical aspect here, the inference and reality to us all is clear. Government datasets are a small percentage of what is openly available and being linked.
True.
Primarily due to access, which has much to do with issues that Government considers important, such as politics, provenance, authority and trust. I am not counting "resellers" in this, as that in itself raises further issues about why some organisations have access to this first hand data, and why the man on the street often doesn't.The transparency issue, since data dissemination process was historically broken for a myriad of reasons. Thanks to the Web and Internet, we can recalibrate as per the global structured goverment data publishing wave. Once this is in full motion, the transparency issues of yore will fade away courtesy of the "citizen analyst" (akin to Web 2.0's citizen blogger).
Government should publish structured data using the deployment formats that work best for them. They are publishers of structured data in this value chain :-) At the warehouse and retail levels we can deal with formats, and the formats that deliver the most value will dominate. Value in this case will ultimately come down to JOIN friendliness (aka. Linkability)/Your reply strengthens my point - that aspect of the equation is technical in nature, and more suited to the purview of the LOD Project and others working on the technical standards side of things. How LOD is linked or structured, the languages they use, REST vs SOAP etc is besides the point - we need to be concerned with how it's going to be delivered via Government channels to the public regardless of the final dominant paradigm that LOD and structured data takes./3) We accept that Linked (Open) Data is the purview of the Linking Open Data W3C Project - there is probably little we can add to the discussion here apart from supporting them in thier own work of IDing datasets that can be linked.In support of this point, e-Government will be as any other entity in this regard, and the methodologies in delivering LOD will not likely differ to the rest of the world or society, much as there is little difference in Web Content Delivery between Government models and Commercial/Public models. In that sense I agree with Thomas 100% when it comes to a technology model. It will be Semantic, and RDF is likely to become the dominant paradigm, if not the only one.5) Open Data therefore is what we SHOULD be focused on - not in the sense of forcing a standard on Gov in terms of Open Data Delivery policy, but in Education and Outreach.The question of non-RDF data consumers is almost moot. Given the time scales we are operating on, it is akin to asking at the start of the first version of HTML "how does hyperlinked content support .txt based users such as BBS systems". Non semantic, non-RDF, pre HTML 5 browsers and technologies will be legacy before we know it, probably while we are still discussing all this. I mean it.This leaves us with two outcomes. The first is that the current user base that Thomas identifies as professional RDF consumers will inevitably drive the conversion of their suppliers data into RDF/XML formats, essentially as a snowball effect. GIS Data is a good example of where this is already happening.//RDF/XML really has little to do with the matter, at best its just a data representation option for an EAV model variant i.e., RDF Data Model.I guess, RDF/XML will continue distract us for as long as it has preeminence in the Semantic Web Layer Cake :-( /
I agree from a technical standpoint, but I stand by my initial plain english version ;)/ //Essence of Linked Data is server the representation requested by the User Agent. If they want HTML you send an HTML+RDFa representation of a resource description etc.. Basically, this is just about using HTTP's in-built prowess, the right way./The second is that as Thomas says, human-readable formats HAVE to be provided - ultimately the user is human, and the transition on the tech side between how the machine reads it, and how it is displayed to the user in a usable, displayable form should be seamless. Ultimately the user should not even realise that they are doing anything but looking at a web page of results that they have asked a server for./The value of HTML representations of resource descriptions remains too under appreciated re. overall demonstration of the real virtues of: HTTP based Linked Data and/or HTTP based Linked Open Data. /
The plain english statement works :-)
So I think we should talk more about LINKs, especially as everyone seems to accept that LINKs make the HTTP based World Wide Web, and the Web is extremely valuable etc. :-)/Yes, but the story is really much shorter and simpler: just get everyone to understand that its about adding LINK granularity to what they already have out on the Web (re. Linked Open Data initiatives). It isn't about some difficult to read or comprehend format called RDF/XML, its just about the evolution of LINKS i.e., we are moving from "Container to Container LINKs" to "Data Items to Data Item LINKs" :-) //This is where I do disagree with Thomas. A Federation of providers is a nice concept, but it is too far off to think about, and will be inevitable in the end so probably doesn't need to be focused on. I believe that the key to overcoming the mistrust issue is three-fold:a) Focusing on educating Governments on WOG methodologies in adopting inter-agency delivery on a National level - ie: promote the creation of the data.gov.* model. The international model is far to scary a prospect for most Governments to contemplate.b) Educating Government on the ROI in making Data open to the publicc) Educating Government in ways in which "clearly marked-off data spaces with a trusted provenance" can still mean open data delivery for all - essentially this already happens whenever data is published, even in a HTML/PDF format - having data in the public domain does not mean giving access to the original uncorrupted dataset./Very well put!
Kingsley
Cheers Chris/ Kingsley // Just some thoughts. Cheers Chris Thomas Bandholtz wrote: //There has been much discussion about *Open* Data in the eGov list thesedays, which is a rather political question.I am currently not so much concerned about openness, more about *Linked* Data, as we have tons of government data with a legal obligation to makethem available to the public (at least in Europe, and especially environmental data), and we are looking for means to do so in the most efficient way. So, among the six items of today's agenda, I find number 6 the most challenging: //6. Discussion: Government Linked Data, Techniques and Technologies [35min] //some considerations: //+ how does linked data support (non-RDF) data consumers? //First of all: Linked Data supports RDF data consumers. Human readable formats should also be provided based on content negotiation. Some providers have dedicated HTML formats, others havenot. Those who haven't depend on some available, general purpose "linkeddata browser".The latest discussion about the state of such tools has been started by http://lists.w3.org/Archives/Public/public-lod/2009Oct/0105.html, and Iam afraid the state-of-the-art of such browsers cannot compete with a well-made dedicated HTML page (how could it). So one might say linked data supports non-RDF data consumers rather badly, but there a two objections:* even non-RDF data consumers benefit from the availability of some linked data which would not be available in the Web at all if notgenerated with D2R (or similar) * even non-RDF data consumers benefit from the extensive andsystematic linkage provided by Linked Data which is rather unusualfor common HTML pages. I think the value of this question is somehow disputable, as - aside form any content negotiation - linked data supports RDF consumers atfirst. These consumers are mostly professionals who depend on governmentdata in order to do their work. So I would rather ask:"How do professional RDF data consumers integrate linked data into theirworking data bases today?"//+ strategies for modelling government data //Well, I would say, the basic model is RDF in this case ;-). We are wasting too much time with efforts on "harmonising" models in a waterfall manner (see http://inspire.jrc.ec.europa.eu/, for example) instead of just publish it somehow. One of TBL's Do's and Don'ts reads:"Do NOT wait until you have a complete schema or ontology to publish data. "http://www.w3.org/DesignIssues/GovDataI do not see any problem about schema diversity. However, we should make use of existing schemas which have proved to work well. For example, theOGC Observation and Measurement XML schema: http://www.opengeospatial.org/standards/om OM is expressed as an XML schema, not in RDF so far. But it expresses perfectly clarified semantics about any kind of measurement data of whatever kind of sensor, including timelines. XSD and URN patterns are some drawbacks of this formalisation, but this could be resolved by a RDF reformulation of the same semantics easily. The most important aspect again is linkage. When expressing what orwhere has been measured, don't use a dumb character string, but link toa reference vocabulary. //VoiD is a good start. I wouldn't overestimate the need for metadata as/+ essential metadata for Government Linked Open Data (eg VoiD) /long as you can access the data itself. Metadata was a great thing informer times when data access was a complex issue, so you would like toknow what you will get before starting the effort to get access to it.If the data itself is linked to reference vocabularies extensively, thedata vs. metadata discussion ends in smoke. //+ expressing rights and licensing information //VoiD can do this. //Government generally is not so amused about the open world assumption,/+ approaches to provenance, authority and trust /they prefer clearly marked-off data spaces with a trusted provenance. I think mistrust can be overcome by federation of providers. Federated agencies can easily state that they trust each provider in this federation. Just set up a domain for such a federation, link to this federation from the data, and to the data from the federation. No problem if anybody is publishing her own possibliy weird statementsabout the same things as long as the federarion does not link to this data.One rather developed case of such a sub-cloud is Linking Open Drug Data(LODD). see http://esw.w3.org/topic/HCLSIG/LODD We might learn from them. //+ using RDF for Statistical Data //Parts of EUROSTAT have been published in SCOVO http://sw.joanneum.at/scovo/schema.html. Even SDMX is apparently moving towards SCOVO. Does anyone see an alternative approach? Looking forward to discussion this afternoon (well, in my time). Thomas (consulting the Federal Environment Agency in Germany)// // ///
-- Regards, Kingsley Idehen Weblog: http://www.openlinksw.com/blog/~kidehenPresident & CEO OpenLink Software Web: http://www.openlinksw.com