Re: Using python to delta-load files into a central DB

2007-04-13 Thread Chris Nethery
Gabriel, I think that would work well. Also, thank you for suggesting the use of filecmp. I have never used this module, but it looks like a much better solution than what I had been doing previously--using os.stat and performing a DB lookup in order to verify that the filename and timestamp

Using python to delta-load files into a central DB

2007-04-12 Thread Chris Nethery
Hello everyone, I have a challenging issue I need to overcome and was hoping I might gain some insights from this group. I am trying to speed up the process I am using, which is as follows: 1) I have roughly 700 files that are modified throughout the day by users, within a separate

Re: Using python to delta-load files into a central DB

2007-04-12 Thread Gabriel Genellina
En Thu, 12 Apr 2007 14:05:15 -0300, Chris Nethery [EMAIL PROTECTED] escribió: At present, users of the separate application can run recalculation functions that modify all 700 files at once, causing my code to take the whole ball of wax, rather than just the data that has changed. Are they

Re: Using python to delta-load files into a central DB

2007-04-12 Thread Chris Nethery
Gabriel, Thank you for your reply. Yes, they are tab-delimited text files that will change very little throughout the day. But, this is messy, antiquated 80s junk, nonetheless. Rows are designated either by a row type or they contain a comment. Each row type has an identity value, but the

Re: Using python to delta-load files into a central DB

2007-04-12 Thread Gabriel Genellina
En Thu, 12 Apr 2007 23:51:22 -0300, Chris Nethery [EMAIL PROTECTED] escribió: Yes, they are tab-delimited text files that will change very little throughout the day. But, this is messy, antiquated 80s junk, nonetheless. Ugh... Rows are designated either by a row type or they contain a