A traditional way to exit a chaotic situation as you describe is
to try to establish a standards committee, invite participation from
suppliers and users of whatever (data in this case), apply for
registration with the International Standards Organization, and organize
meetings, draft and circulate a proposed standard, etc. A statistician
who had published maybe 100 papers and 3 books told me that his work on
ISO 9000 (I think) made a larger contribution to humanity than anything
else he had done. Work on standards is one of the most boring, tedious
activities I can imagine -- and can potentially be the most impactful
thing one does in this life: If you have an ISO standard number for
something, people who are starting something new may find it and follow
it. People who are working to upgrade something may tell their
management, "Let's follow this standard." Customers sometimes ask their
suppliers, "If you follow the standard, you might get more customers."
I think you could get support for such a standard effort from the
American Association for the Advancement of Science, the American
Economics Association, the American Statistical Association, and many
other organizations, including many on-line science journals that today
pressure authors of papers to put the data behind their published paper
in the public domain, downloadable from their web site, etc.
IMHO.
Spencer
On 1/13/2012 3:39 PM, Benjamin Weber wrote:
The whole issue is related to the mismatch of (1) the publisher of the
data and (2) the user at the rendezvous point.
Both the publisher and the user don't know anything about the
rendezvous point. Both want to meet but don't meet in reality.
The user wastes time to find the rendezvous point defined by the publisher.
The publisher assumes any rendezvous point. As per the number of
publishers, the variety of the fields and the flavor of each expert,
we end up in today's data world. Everyone has to waste his precious
time to find out the rendezvous point. Only experts do know in which
corner to focus their search on - but even they need their time to
find what they want.
However, each expert (of each profession) believes that his approach
is the best one in the world.
Finally we have a state of total confusion, where only experts can
handle the information and non-experts can not even access the data
without diving fully into the flood of data and their specialities.
That's my point: Data is not accessible.
The discussion should follow a strategical approach:
- Is the classical csv file (in all its varieties) the simplest and best way?
- Isn't it the responsibility of the R community to recommend
standards for different kinds of data?
With the existence of this rendezvous point the publisher would know a
specific point which is favorable from the user's point of view. That
is missing.
Only a rendezvous point defined by the community can be a 'known'
rendezvous point for all stakeholders, globally.
I do believe that the publisher's greatest interest is data
accessibility. Where is the toolkit we provide them to enable them to
serve us the data exactly as we want it? No, we just try to build even
more packages to be lost in the noise of information.
I disagree with a proposed solution to have a maintained package or a
bunch of packages which just combines connections to the existing
databases and keeping them up to date. It is a question of time when
the user will be lost there. Such an approach is neither feasible, nor
efficient.
We should just tell them where we would like to meet.
Benjamin
On 14 January 2012 04:58, Brian Diggs<dig...@ohsu.edu> wrote:
On 1/13/2012 2:26 PM, MacQueen, Don wrote:
It's a nice idea, but I wouldn't be optimistic about it happening:
Each of these public databases no doubt has its own more or less unique
API, and the people likely to know the API well enough to write R code to
access any particular database will be specialists in that field. They
likely won't know much if anything about other public databases. The
likelihood of a group forming to develop ** and maintain ** a single R
package to access the no-doubt huge variety of public databases strikes me
as small.
I agree. The more reasonable model is a collection of packages, each of
which can access a particular data source.
However, this looks like a great opportunity for a new CRAN Task View. The
task view would simply identify which packages connect to which public
databases. (sorry, I can't volunteer)
A CRAN Task View would be well suited for this. I have tagged these sort of
packages on crantastic with the "onlineData" tag when I happen to notice
one, but I have not made a concerted effort to find all packages. A Task
View would be even better.
http://crantastic.org/tags/onlineData
-Don
p.s.
I can mention openair as a package that has tools to access public
databases.
Tagged it.
--
Brian S. Diggs, PhD
Senior Research Associate, Department of Surgery
Oregon Health& Science University
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Spencer Graves, PE, PhD
President and Chief Technology Officer
Structure Inspection and Monitoring, Inc.
751 Emerson Ct.
San José, CA 95126
ph: 408-655-4567
web: www.structuremonitoring.com
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
