I've been trying to do a small research paper on IPv4 address depletion and the future of IPv6, yadda-yadda, so I got ahold of some raw data from ARIN, LAPNIC, APNIC, AND RIPENCC. This came in the form of txt/csv files via ftp.
Using Linux was a big help at the beginning, using sed to massage the data somewhat and combine it all into one file. The result was a txt file with 63,260 rows in 7 columns -- about 3 MB total. I fully realize this is into the realm of a database, but I wanted to do some analysis using the Datapilot functions.
Importing into Calc was fairly straightforward and the file loaded in just a few seconds. Now the fun begins. Tried to use the Datapilot but it simply choked and came up with an error. This is 1.9.91 on FC3. Then I tried to simply save the file in the ODS format. That took *over* *thirty* minutes! I then saved it again in xls. Maybe 5 minutes or so for that operation. Screwed around with it for the better part of a day. Everything I tried to do was soooo painfully ssslllloooowwww. Gave up.
Rebooted into XP. Opened up Excel 2000. Loaded the original data file. Added a column using an array formula. Fired up the Autopilot. Got my analysis done and made a nice chart. Saved my work. Total time -- maybe an hour. Saving the xls file when I was done took just a few seconds.
This is not meant as a troll, just an observation. Like I said, I really love using Writer and Draw. But the xml file format for spreadsheets is really unusable for a large sheet. Out of curiousity I renamed and unzipped the ods file. About 50 MB. The xls is about 10 MB (with Autopilot sheet an Chart) and the zipped ods is about 600 KB.
I couldn't care less about the space; I have 100 gig on this system. But notice what happened: the 3 MB csv was transformed into over 48 MB in the ods format before zipping. That's a 16 to 1 ratio.
I understand the importance of open formats, but we really need to come up with something better than that. For small sheets it really doesn't matter that much, but with bigger data sets it is simply unusable. And it's not just OOo. I tried the same thing with gnumeric. Once again, saving into their xml format was just painfully slow as well.
Comments?
Rod
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
