Doug - On June 07, 2012 6:08 PM Doug Chamberlin wrote: > Yes. On further thought I think I would make the whole ETL process a > two-pass affair. First pass check for existence of all needed entities > and creates ones that are missing. Second pass should do the inserting > proper without errors.
That is actually an interesting idea that I hadn't thought of. We are dealing with fairly large files - 250k to 900k lines in file, so I'm not sure if the overhead of doing two-pass processing can be absorbed, but it's an option to try. As to your other idea of processing the FK entries in a seperate transaction - that is the first idea that I had, but we wound up with serious issues when we last tried that - it's been about a year and a half or so, and I don't remember the specifics right now. Maybe I'll go back and check that one more time... Thanks. ------------------------------------ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Visit http://www.firebirdsql.org and click the Resources item on the main (top) menu. Try Knowledgebase and FAQ links ! Also search the knowledgebases at http://www.ibphoenix.com ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Yahoo! Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/firebird-support/ <*> Your email settings: Individual Email | Traditional <*> To change settings online go to: http://groups.yahoo.com/group/firebird-support/join (Yahoo! ID required) <*> To change settings via email: firebird-support-dig...@yahoogroups.com firebird-support-fullfeatu...@yahoogroups.com <*> To unsubscribe from this group, send an email to: firebird-support-unsubscr...@yahoogroups.com <*> Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
