On 11/13/07, Landon Blake <[EMAIL PROTECTED]> wrote:
Puneet,
You wrote: "Should be easy to transition to. By building the new
format
on the
structure of the Shapefile format, and *in fact*, calling it "open
shapefiles" or some such thing, we indicate from its name that the
transition is not that revolutionary but is evolutionary. This,
hopefully, will bring some name-familiarity, and make the transition
less scary."
I really think you are going to run into problems using the
"Shapefile"
as part of the trademark or name for any product not sold by ESRI. I
strongly recommend against this move. Let people adopt the
implementation of your idea for its merits, not for name recognition
that comes from another product line.
Good enough point to keep in mind, but not to get hung up over enough
to entangle us. Suggestions for names of the data format can be a
project in itself. "open spatial data format" or its variations could
be chosen. Still, point taken.
You wrote: "ANSI standard C is still
that magic common denominator that compiles and works predictably on
most number of systems. I have a lot against Java, but those who love
Java should definitely work on tools for accessing and working with
this new format as it would only make the format more widely used and
adopted."
It sounds to me like you are really describing a tool. File formats
are
written in a binary encoding or text, not in a programming
language. If
you are designing a tool you can choose the programming language of
your
choice, but be aware that this will limit the developers that adopt
the
tool. This will be the case no matter what language you choose to
use,
whether it is C, Java, or something else.
If, in contrast, you are creating a file format, then programming
languages shouldn't really matter. Binary and text data can be
accessed
by almost all programming languages.
I think you need to decide if you want a tool or a data format. It
sounds like you are shooting more for a spatial database written in
the
C programming language that uses some form of the ESRI Shapefile as
its
underlying data storage mechanism. To me that is a tool or piece of
software, not a format. But maybe I don't completely understand your
goal.
well, I am, frankly confused.
I was quite convinced I wasn't describing a "tool" but was describing
a "format." Of course, to describe the format, I positioned it on the
"format" (the SQLite-compatible format) used and popularized by a
"tool" (SQLite, the library, which happens to be written in C). In my
mind, having the data format based on SQLite *format* for its
relational attribute handling was the real winner. In that sense,
perhaps I conflated the format and the tool. I am not well versed in
these things to I am probably already walking on thin ice, but that
shouldn't stop others.
So, forget that I mentioned C and Java... let's just concentrate on a
way of laying out data on a disk that is not too dissimilar from how
Shapefile data are laid out, except that we utilize the
SQLite-compatible binary format for relational data handling, so that
SQLite-enabled spatial tools can access this new format.
And, put this format into public domain.
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of P Kishor
Sent: Tuesday, November 13, 2007 8:35 AM
To: OSGeo Discussions
Subject: [OSGeo-Discuss] Re: idea for an OSGeo project -- a new,open
data format
Thanks everyone, for responding. Here is my "groundwork."
The new format --
- Should be fast. SQLite is plenty fast, and anything that simply
"extends" the Shapefile format to inject relational capabilities
should be pretty fast. It should definitely be faster than a
geodatabase format (such as PostGIS/ArcSDE) and perhaps even faster
than Shapefiles especially while accessing attribute data. DBF is
sequential, and searching for textual information is particularly
expensive. SQLite has been tuned to excellence. I have been working
with it for a few years now, and it really is an amazing product,
development community, support, and capabilities. That it is in
public
domain makes for a transfat-free icing on the cake.
- Should be unencumbered by licenses and copyrights. Ideally, the new
format could also be put back into public domain. We want to remove
all encumbrances to encourage rapid and wide adoption.
- Should be a single file. Well, some like multiple files and some
like single files. We can achieve both objectives by using a
tar-gzipped packaging such as Apple tends to use for much of its
stuff
(for example, its Pages wordprocessor uses a tgzipped xml file along
with other resources for icons and pictures and stuff). Or, if speed
is going to be affected because of gzipping and gunzipping, just a
package format (I have no idea if this is a Unix thing or a Mac OS
thing -- we, in the Mac world, call them packages... they appear like
files in the Finder, and like directories in the shell).
- Should be easy to transition to. By building the new format on the
structure of the Shapefile format, and *in fact*, calling it "open
shapefiles" or some such thing, we indicate from its name that the
transition is not that revolutionary but is evolutionary. This,
hopefully, will bring some name-familiarity, and make the transition
less scary.
- Frank mentions SQLite's lack of datatypes as an issue -- I guess
that is a matter of preference. I personally quite like that freedom
as it gives me, the application developer, complete control over what
goes where. SQLite actually does have now a few datatypes that it
respects, but doesn't complain about. Since all users will be
accessing the data via an application, as long as the application is
well defined, it should be fine.
- SQLite excels at one thing that it has been entrusted to do --
retrieve data that it has been entrusted with at extremely fast
speeds, and maintain ACID data integrity in case of a programmatic
catastrophe. The transactions themselves are worth their price of
admission, which, happily, happens to be zero.
- Langdon mentions Java support -- well, yes, use/work on SQLite
JDBC.
I have been using it for a few days now and find it to be a pretty
competent conduit. Extend it, spatialize it. ANSI standard C is still
that magic common denominator that compiles and works predictably on
most number of systems. I have a lot against Java, but those who love
Java should definitely work on tools for accessing and working with
this new format as it would only make the format more widely used and
adopted.
Ok, enough for now.
On Nov 13, 2007 8:52 AM, P Kishor <[EMAIL PROTECTED]> wrote:
So, I am thinking, Shapefile is the de facto data standard for GIS
data. That it is open (albeit not Free) along with the deep and wide
presence of ESRI's products from the beginning of the epoch, it has
been widely adopted. Existence of shapelib, various language
bindings,
and ready use by products such as MapServer has continued to cement
Shapefile as the format to use. All this is in spite of Shapefile's
inherent drawbacks, particularly in the area of attribute data
management.
What if we came up with a new and improved data format -- call it
"Open Shapefile" (extension .osh) -- that would be completely Free,
single-file based (instead of the multiple .shp, .dbf, .shx, etc.),
and based on SQLite, giving the .osh format complete relational data
handling capabilities. We would require a new version of Shapelib,
improved language bindings, make it the default and preferred format
for MapServer, and provide seamless and painless import of regular
.shp data into .osh for native rendering. Its adoption would be
quick
in the open source community. The non-opensource community would
either not give a rat's behind for it, but it wouldn't affect
them...
they would still work with their preferred .shp until they learned
better. By having a completely open and Free single-file based,
built
on SQLite, fully relational dbms capable spatial data format, it
would
be positioned for continued improvement and development.
Is this too crazy?
--
Puneet Kishor
_______________________________________________
Discuss mailing list
Discuss@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/discuss
Warning:
Information provided via electronic media is not guaranteed against
defects including translation and transmission errors. If the
reader is not the intended recipient, you are hereby notified that
any dissemination, distribution or copying of this communication is
strictly prohibited. If you have received this information in
error, please notify the sender immediately.
_______________________________________________
Discuss mailing list
Discuss@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/discuss
_______________________________________________
Discuss mailing list
Discuss@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/discuss
