We did this at one client I was working in, whose name will be withheld
for various reasons. We had all the data in another database, but like
you've run into, we didn't have the time to get all the data in within a
timely manner.
So we used ARSPerl to create and run multiple perl scripts that would
take segments of the data and push it into Remedy. We had Remedy
configured with multiple instances connecting to the same database, and
we had some major hardware in place. I think the most important part
for you right now will be the multiple instances of ARS, which will in
effect make escalations multi-threaded. The only problem is that I
don't know if you can set up multiple instances of ARS using demo
licenses or if you have to pay for licenses, because that can get
expensive.
Shawn Pierson
-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of William Rentfrow
Sent: Monday, October 15, 2007 12:35 PM
To: arslist@ARSLIST.ORG
Subject: CTM:People large data imports
**
On my current project we will have to import about 1,000,000
CTM:People records every week (give or take). These will be "new"
records every time because the only unique identifier we have from the
external systems is an identifier like SSN that we can't put into our
system (that's a policy decision). We're checking on ways to do updates
but it's still messy due to the item below.
The actual import is going to be automatic and should follow
this process:
1.) Escalation deletes all relevant CTM:People records
2.) File is automatically put on the Remedy server with people
information
3.) Escalation fires a Run Process that imports the data into a
staging form.
4.) Escalation fires and sets a flag field to "Process Now" on
all records in the staging form
5.) Filter on "modify" on the staging form pushes to CTM:People
Here's the problem - we're using decent hardware running on
Solaris 9, etc - yet the push to CTM:People takes 1.5 seconds per record
due to all of the validation that takes place.
I have to do 1,000,000 records a week - and there are only
604,800 seconds in a week....also, it appears using direct SQL loads
won't work due to all of the back end set fields (I can get around all
of those except the GUID - which could be set by another
escalation.....and then we are back at this same problem).
If you've done large data imports to CTM:People I'd really like
to know how you went about loading the data efficiently.
William Rentfrow, Principal Consultant
[EMAIL PROTECTED]
C 701-306-6157
O 952-432-0227
__20060125_______________________This posting was submitted with
HTML in it___
Private and confidential as detailed <a
href="http://www.sug.com/disclaimers/default.htm#Mail";>here</a>. If you cannot
access hyperlink, please e-mail sender.
_______________________________________________________________________________
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org ARSlist:"Where the
Answers Are"
