Hi Folks --
Been working on this issue for a while, and not making much headway, so I
thought I'd throw it out to the list and see what wisdom came back :)
Problem: I'm trying to find a way to store LOTS of scientific data (mostly
32-bit integers and floating point numbers) in a database for analysis work.
The instrument(s) we're working with generate very large volumes of numeric
data, roughly 30GB per day.
What I'd like to do is have a method read through the data files produced,
store the data in tables, making it easy to subset and index the data. OK,
doesn't sound too bad off the bat.
Here's the issue: Is there a way to store data in BINARY numeric formats
into the database (without converting to strings first)?
In two or three of my tables, rows contain 2500-3000 individual numbers, which
would be a logistical nightmare to throw into a query string, not to mention potential
loss of accuracy from rouding errors in floating point numbers.
I thought that LOAD DATA INFILE was the next solution, but it appears to, when
using fixed-row format, is that it uses the DISPLAY sizes of the fields,
rather than the byte storage requirements for them.
Anyone else out there dealing with large volumes of numeric data?
Thanks in advance,
Ken
===========================================================================
"Diplomacy is the weapon of the Civilized Warrior"
- Hun, A.T.
Ken Gieselman [EMAIL PROTECTED]
System Administrator http://www.endlessknot.com/~kgieselm
Endlessknot Communications http://www.endlessknot.com
===========================================================================
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]
