If analyzing a flat file is what is taking time, then I'm afraid there is nothing you can do to speed up that..
Having said that.. what sort of analysis does it perform? Does it compare the information in the flat file against information that may already be within the database? Is that what usually takes time? Joe D'Souza -----Original Message----- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] Behalf Of Peter Romain Sent: Wednesday, October 03, 2007 3:37 AM To: arslist@ARSLIST.ORG Subject: Re: EIE vs. arimportcmd flat file import performance Joe, If you look at the log files generated it: - parses the flat data flat file - gets the .tbl file containing the data mapping definition - does some analysis of each field in the data mapping definition - does some analysis of each field in the CMDB that is referred to - creates the records in the CMDB The analysis takes place for each record in the data file. It's a few months since I was last involved in this but the precessing seemed unnecessarily thorough at the time. Interestingly, we could have used CMDB web services which was much faster (similar to the Import Tool) Cheers Peter > Peter, > > When you state 'see all the processing it's doing to get the data mapped', > what exactly do you mean. What does it do? > > Searches against tables with large volumes of data? > Run external scripts / commands / utilities and wait for their output? > > Joe D'Souza > > -----Original Message----- > From: Action Request System discussion list(ARSList) > [mailto:[EMAIL PROTECTED] Behalf Of Peter Romain > Sent: Tuesday, October 02, 2007 1:16 PM > To: arslist@ARSLIST.ORG > Subject: Re: EIE vs. arimportcmd flat file import performance > > > I did some initial testing of EIE into CMDB 2 and found it much slower > (about 1/3 the speed). When you look at the logs and see all the > processing > it's doing to get the data mapped from the flat files then you can > understand why! > > However, by careful splitting of the data it is possible to use multiple > parallel EIE data imports to speed things up. It is also possible to run > multiple EIE daemons and direct the data at these to further improve > performance. There is a white paper on EIE performance tuning. > > I guess it comes down to how many records, where they will go (CMDB is > particularly slow), how much time you can allow and how complex you want > the > import to be. > > I doubt you'll get away without testing. > > Cheers > > Peter > > >> We are strategizing a way to import a large number of records on a >> routine basis. I'll just skip all the details since they really are not >> relevant to the core question: >> >> Has anyone compared import speed between the EIE and using the "old >> school" arimportcmd command line process in order to import a flat file? >> I am just curious to see if there is a big advantage in speed either >> way. There certainly advantages to both tools for doing this particular >> data import but I have no idea how the speed compares and I don't have >> the EIE installed on this server (and I'd rather not if I don't have >> to). >> >> William Rentfrow, Principal Consultant >> [EMAIL PROTECTED] >> C 701-306-6157 >> O 952-432-0227 No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.488 / Virus Database: 269.14.0/1046 - Release Date: 10/3/2007 10:08 AM _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org ARSlist:"Where the Answers Are"
