Hey Cameron, all,
Cameron, you recently asked me to join your letter from the OSGeo to
the OGC Members regarding the adoption of the proposed "ESRI
GeoServices REST API" as an OGC standard.
http://wiki.osgeo.org/wiki/Geoservices_REST_API#Open_Letter_to_OGC_and_voting_members
Thanks. Your request has forced me to define my position, which has
been a huge waste of my time, but was probably worth doing.
My response has two sides, my position on the debate and my decision
to participate in the letter.
The first is my position on the actual proposed standard.
The pros:
--------
* The OGC should be in the business of developing good standards,
not of choosing which standards should be implemented.
* The proposers of the document want to make a standard and have
followed all the rules of the consortium. The work of any such
group of members deserves serious, good faith consideration.
* The need for an integrated suite of services using simple data,
which is addressed (partially) by the document, is real. The
proposed document is pushing the OGC on this issue.
* The proposed document could be useful to a number of people.
* The proposed document is not significantly more broken than the
existing standards of the OGC. As an author of standards at the
OGC, I know how totally impossible it is to write a good standard,
so the weaknesses in the existing document seem more acceptable.
The cons:
---------
* The OGC is, de-facto, in the position of recommending the
interoperable standards for geospatial services. The proposed
document is not good enough, not widely enough implemented, and
not publicly supported enough, to be considered at the same level
as existing standards.
* Adopting a standard implies a desire to maintain the standard
but the desire to support this approach by the OGC membership is
limited. The lack of collaboration on this version bodes ill for
the future.
* The overlap in functionality between the proposed services and the
existing services, notably with the ongoing work to modularize the
existing services, is almost 100 percent. However, compatibility is
low.
* There is already a published document:
http://www.esri.com/library/whitepapers/pdfs/geoservices-rest-spec.pdf
so there is no need for the document to be adopted as an
OGC Standard merely for interoperability with the ESRI
implemetation.
* The document, as a new, separate effort, repeats mistakes which were
made and since solved by the other services.
* The document focuses on the past (notably with backwards
compatibility and use of only GET/POST) not on the future.
* The document needs a comprehensive editorial rewrite. (see at end)
The problem for me, is that both simple answers are bad.
A simple acceptance of the standard would introduce a new set of 'OGC
approved' open services. The OGC approval might enable governments to
buy a XXXX-new-name-here-XXXX solution instead of a W*S or a S*S
solution. (On that, I am inclined to let them make their own
mistakes.) The path forwards towards harmonizing the services is
unclear. There is little good will towards the standard on behalf of
the membership. Fixing this document in addition to fixing the W*S
services will be a pain.
Simply rejecting the solution would be bad for the OGC. It would place
the OGC in the position of picking winners and losers in the standards
business. It would mean that the OGC is stuck on the project of fixing
the W*S standards to meet some nebulous future functionality without
having any path to get there. It would discourage innovation and
progress.
Is there a third way?
Well, actually, there is. The natural consequence of either decision
is a renewed commitment towards trying to build this thing that we
want, an integrated suite of simple services built using simple, well
defined resources, accessible and usable using the core HTTP verbs,
and discoverable through link following. That's the way forwards and
why I have wasted a chunk of my life on WMS-Next.
My personal conclusion, then, is that the vote does not concern me.
I suspect the vote is mostly of interest to the commercial entities,
groups whose self-interest is so destructive everywhere because it
does not place user freedom and action first and foremost. Acceptance
will have several net negative impacts on me but that's life, eh?
For me, the path forwards is clear: make WMS-Next kick ass. The vote
is merely a waste of my time.
The second side to my response regards the actual letter.
The pros:
--------
* The letter comes from the Free Software community which I think
could have a voice on the matter.
* The letter was started by someone who has my respect.
* The letter re-enforces the link between OSGeo and the OGC.
* The letter expresses many concerns which I share.
The cons:
--------
* The letter comes from OSGeo, a group with which I have an
ambivalent relationship from being a second class citizen and
from its focus on 'open source' rather than on freedom.
* As a member of the OGC, I have a direct voice there (albeit again
as a second class member without a vote) so it makes more sense
for me to act directly at the OGC than indirectly through OSGeo.
* The letter is only rejection of the proposal without offering an
alternative way forwards.
* The critique of the letter is not yet fully formulated making it
hard to decide if I actually agree with it.
* The critiques do not inspire me. I suspect none of you has actually
read the proposed document, nor have any real analysis to offer of
it. (EDIT: actually, a new wave of criticisms focused on 'imagery'
seem to come from folk who actually have read the document. Cool!)
* Some the concerns are wrong. (It seems, this is simply due to lack
of knowledge of the details, a minor issue ultimately).
My personal conclusion then is that I should not be trying to express
myself on the matter through an incompletely formed statement of the
OSGeo community.
Therefore, I will not add myself to your letter.
Nonetheless, thanks for your effort, thanks for asking, and thanks for
prodding me to sit down and write this; given the time I have taken,
it is clear that I wanted to make myself do it.
cheers,
~adrian
A Quick Critique of the proposed standard document.
---------------------------------------------------
1. The name.
The document currently uses a name that is confusing in all
discussions on the matter, both internally at the OGC and externally.
(Yesterday, the Standards Working Group proposing the document agreed
to accept proposals for new names; if they can find a good
alternative, they may adopt it.)
2. The design.
The document does not show an overall, coherent design to the service
suite. This is mostly because the design reflects a suite that evolved
piecemeal at ESRI rather than one designed from the start to meet the
needs of a broad swath of users. This is also due because the experts
at the OGC did not contribute to an improved design for a multitude of
reasons.
The design of the service which does exist is focused on yesteryear.
As an apology for the existing design, section~6.2.2 of part 1, Core
says that the only HTTP verbs used are GET and PUT because "the API
was originally developed several years ago" but that's okay because it
enables support for "Microsoft Silverlight" and "Adobe Flex" two
standards which are dying rapidly. In 2013, a new standard ought to be
focused on the next 10 years not the past ten.
3. The writing.
The document is poorly written.
3.1 The document is poorly introduced.
The document should start with a discussion of the overall design of
the suite of services and what each individual service provides within
that suite. This would make it easy for a user to understand the focus
of the suite. (The note between requirements~3 and requirement~4,
stating that the spec makes no requirements on clients, is really part
of the scope of the document and should be front and center.)
The document instead starts with a whole section on REST.
1. Scope
The GeoServices REST API provides a standard way for web clients to
communicate with geographic information system (GIS) servers based
on Representational State Transfer (REST) principles.
The scope claims that the document identifies resources and a way to
use "structured URLs" to exchange those resources between clients and
services. If that were true, I would expect
(1) a list of resources at least as an example, and
(2) an example of the structure and structuring principles of
the URLs.
Section~6 stats this process. Section~6.1 introduces the concept of a
root URL for each service and mixes in that that root URL is also a
catalog and that the hierarchy offers resources. This all happens in a
single, overly compact paragraph. This section needs an editorial
rewrite for clarity separating
(1) the end point,
(2) the catalog,
(3) the URL hierarchy,
(3) the resources, and
(4) interaction of clients with the URLs through the exchange
(get/post) of the resources to the different URLs.
Unfortunately, section~6 then decays into a meandering discussion of
REST which mentions the buzzword a lot but does not discuss the
aspects of REST design that have been met by the suite and those that
have not. If REST is a form of design which offers certain benefits,
OGC standards should be discussing the benefits achieved not the buzz
word. To what extent are the resources cachable and how do clients and
servers agree on that? To what extent are the endpoints and actions on
those endpoints discoverable, and how? Those are the issues of RESTful
design.
Section~6.3 discusses Resources but immediately then states:
Each resource has a unique URL. Resources MAY support parameters.
which is a discussion of the URL hierarchy. (And 'Resources' do not
'support parameters'! Services may 'handle requests containing
parameters' if that is what is meant.) Then the standard introduces a
new idea 'Controller Resources' which are able to 'edit' and 'query'!?
A web search does not reveal any formal discussion of 'Controller
Resource' so this new type of resource probably deserves some kind of
formal introduction.
Section~6.3.3 is not really about resources but the fact that the URL
hierarchy is not discoverable from the Resources themselves.
Section~6.3.4 does give a list of resources, but since 'image' is not
part of it but will be used by the Service for Map Images, I surmise
that the list is of the things 'we have designed a JSON representation
for' rather than a list of 'the resources identified and exchanged in
the service suite' which is what I expect in an introductory design.
This introduction therefore needs a substantial editorial rewrite to
be effective in its goal of explaining what the suite of standards
actually does.
This criticism is not limited to the core document.
The Service for Maps document also starts without setting out the five
core elements (end pt, catalog?, URL hierarchy, resources, and
interactions).
Section~6 of the Service for Maps part, in the middle of the second
paragraph, introduces a new concept, with capitalized letters:
The Map Service Root resource includes a tables property ...
without giving us any clue what this is. It "provides basic
information" (perhaps it is a data structure?) but it also "supports
several operations" (maybe it's a magical 'Controller Resource'). To
me, this just reads as 'the authors of the spec are confused and
imprecise in their written language'.
Then, in the figure of 'Resources,' there is actually a resource
called 'All layers and tables'. Sorry, but this reader needs some real
help to understand what that resource might be.
3.2 The document requirements are poor.
The very first requirement, Requirement~1 of the Core document, states:
Req 1 If a request uses the HTTP GET method, the processing of the
request by the service SHALL be safe and idempotent as specified
by RFC 2616, 9.1.
This is a *ridiculous* requirement, completely untestable, and
explicitly what the Mod Spec should be preventing with its attempt to
require all injunctive language to be accompanied by formal tests. The
formal test indeed shows that the requirement is untestable.
Inspect the documentation of the implementation to identify, if
requests specified in the GeoServices REST API standard that
support the HTTP GET method, are all safe and idempotent, i.e.
free of side- effects, as specified in HTTP (RFC 2616, section
9.1). Alternatively, ask the developer of the the service.
Note that it is not possible to run automated tests against a
service to verify conformance with the requirement (and IETF
provides no test for conformance with HTTP either). In the absence
of such tests, a statement of the developers of the service is
sufficient indication that the requirement was addressed.
So essentially this requirement is met if the developers say 'yeah, we
thought about it.' In other words, requirement 1 of the whole shebang
is a recommendation. Contrast this with requirement 2 which has a nice
test defined for it and is a good, solid requirement.
Again, broadening the critique to the Services for Maps, its first
requirement states:
Req 1 The Map Service Root resource SHALL accept requests that
conform to the URI template in Table and use any HTTP method
identified in the same table.
but the test is then completely irrelevant. At the protocol level, all
web services 'shall accept' request messages, so we are not discussing
that. A functional test for this 'shall accept' is to construct all
valid requests and see that the service never returns an exception to
those requests. Even worse, the first test is a catch all 'please test
the service here' test. This is *not* a well written specification of
the tests for the injunctions of the Service for Maps.
The requirements need a complete, systematic review to see that they
work, they are testable, and have good tests.
4. The document has numerous other issues which need resolution
* There is no discussion of the overlap of the f= parameter and of the
HTTP format header. What happens if they conflict?
* The lack of full integration of the four core HTTP verbs, notably
DELETE and PUT, exists merely for backwards compatibility not for
forwards progress.
* Requirement~3 blocks forwards progress and experimentation with no
explanation as to its importance.
~~~~This goes on and on. This is the work for an EDITOR.
I am not interested in doing that work.
