what do you think about foreign data wrappers getting CSV file table I/O? - I had thought that CSVQL db could be implemented completely with small amount of memory and file I/O, line at a time. EOL detection would be needed. can be: CR, LF, CR+LF. sometimes beginners get it backwards (LF+CR), but it's stll detectable because it's an immediate sequence detectable by state machine or while loop. it should be written as CR+LF because of standards compliance X2J4579 - that means go look for it (I have not found it yet). while (!feof(...)&&(ch=='\r'||ch=='\n')) { do { if (1!=fread(&ch, 1, 1, ...)) { ch=0; break; } if (ch!='\r'&&ch!='\n') { break;//process non-EOL character } //at this point, it's an EOL character } while (true); or while (!feof(...)&&ch=='\t') { if (1==fread(&ch, 1, 1, ...)) {//read 1 character } } this code could be read in chunks using a buffer. the last chunk would need to be handled as a special case, since if it exists, it exists as <full buffer size. try %
- the microsoft patented CSV would be required for implementation. it handles special data with commas and double-quotes in them - tab-separated I/O would be nice as well. - you could see >2,000,000 rows, so don't limit it. try GCC's fopen64/fsetpos,fgetpos/fpos_t/fclose, other vendors use plain fopen and that works with >=64-bit file sizes. and it's fast. avoid 32-bit fseek/ftell. - indexing could be file/RAM array of uint64_t-like file pointers convertable to fpos_t if needed. - biggest needed feature is an easier-to-use ALTER TABLE RENAME. a memorable alternative/alias would be simply RENAME COLUMN columnName TO newColumnName. -- ====== Jim Michaels <jmicha...@gmail.com>