----- Original Message -----
Sent: Saturday, December 08, 2001 10:18
AM
Subject: RE: using XSLT to generate
SQL
It might be better to process the XML file directly with SAX,
DOM or JDOM apis.
Given the amount of data, and logic to implement, I think you
are better off writing this in Java than XSL.
Chris Raber, Systems Engineer, AvantGo Inc.
http://www.avantgo.com/
-----Original Message-----
From:
ishmael [mailto:[EMAIL PROTECTED]]
Sent: Saturday, December 08, 2001 5:04 AM
To: [EMAIL PROTECTED]
Subject: using XSLT to generate SQL
Hello all,
I'm new to both this list, XSL, and Xalan so forgive any
mistakes or
ignorance on my part.
The Problem:
Basically, I have a large
amount of data that exists in an XML format (w/
DTD).
I need to get the XML data into a relational database which has an
existing schema. From what I've read so far it seems like I
can use XSLT to
generate SQL statements that then get
sent to the database via JDBC or
whatever. The real
problem is the amount of data I have to deal with. To
start off there will be ~ 10 million records followed by weekly updates
of ~
10k records or so. Each XML file will have to
generate many INSERT, UPDATE,
and DELETE SQL
statements all wrapped in a transaction, so that the whole
file can be rejected if something goes wrong.
Question #1: Is Xalan or even XSLT right for this job? It
seems like this is
a fairly common problem there must
be existing tools.
Question #2: Does anyone have any comments or suggestions for
dealing with
large data sets and databases?
Optimizations, things to watch out for and
what not.
Does Xalan have to load the whole XML file into memory in order to
work with it?
Question #3: Another potential problem is that while parsing
the document we
won't know if an entity should
generate an UPDATE or an INSERT without first
checking
the database to see if that record already exists. How will Xalan
handle this?
Thanks in advance for the abundance of wisdom, charity, and
encouragement
that I'm about to receive from the good
people of Xalan land. =)
-ishmael
