Some good reasons why homegrown stuff can be faster:
- if you already know exactly what data needs replicated, you don't need timestamp/diff/checksum/trigger code to figure it out
- if your transactions can't conflict by design you don't need to worry about conflict resolution
- you get to choose the best protocol for the job: chatty as you wanna be on a SAN, compressed on dialup, 1 big block on a reliable network, chunked on an unreliable one.
A quote from a paper on WAN synch sums it up:
"Most of the state-of-the-art commercial database systems provide some level of database replication. However, in all cases, their solutions are highly tuned to specifc environment settings and require a lot of effort in their setup and maintenance."
http://www.cnds.jhu.edu/pub/papers/cnds-2002-4.pdf
"On the Performance of Wide-Area Synchronous Database Replication"
I had personal experience with an Oracle based system where none of the DBAs involved (ours, the customers, oracle own reps - all certified) could figure out how to stop replication bombing out. IIRC it was filling acres of disk space up creating and destroying rollback segments. It was also quite chatty - it seemed to be designed for high bandwidth environments, which wouldn't suit Ranjith.
In our case, we were not replicating transactions that would need reconciliation, only farming out copies of new data. Simply mirroring data files with rsync would have been a better option, if the company hadn't been obsessed with oracle...
http://rsync.samba.org/index.html
If Ranjith really wants to go with an XML solution, he could do worse than read the design of SyncML:
http://www.syncml.org/technology.html
which, while it is aimed at PDAs, is at least designed to operate reliably on slow links. The comparison of SyncML and other sync algorithms in this thesis is interesting too: http://ipsit.bu.edu/nislab/theses/ska/thesis_2.html
XML is mainly useful in DB import/export if the databases are not the same product and/or version. DBUnit contains possibly useful code that lets you import/export an XML format spanning multiple tables for any JDBC database:
http://dbunit.sourceforge.net/howto.html#extract
of course there's also Sun's XMLRowSet.
I dont think web services are going to be the answer, but hopefully this will give Ranjith something more to read.
-Baz
Anne Thomas Manes wrote:
What are the reasons why that database links don't work? What is the volume of data that needs to be transferred? My initial reaction to this scenario is to use database synchronization technology. I probably wouldn't consider using XML or Web services.
Anne
-----Original Message-----
*From:* Ranjith Kodikara [mailto:[EMAIL PROTECTED]]
*Sent:* Wednesday, February 05, 2003 12:02 AM
*To:* [EMAIL PROTECTED]
*Subject:* Data Transfer with XML between two DB's- a Real world Case
My customer has a requirement like this. There are several locations
of that organization (my customer). They have an Oracle database in
each of their locations. There is a central server. When daily
transactions are processing, they update their location databases
and at the end of the day, they have to transfer those data to the
central server. Currently we are doing this with Oracle database
links and due to so many reasons this is not 100% successful.
Communication is through internet dialup connections (we can�t tell
them to upgrade those lines)
We are hoping to go for XML technologies in order to do this data
transfer. I have knowledge on XML technologies, but not practical
experience. But in this case there is no one in the central server
to do any thing. People in locations should be able to transfer data
of that particular day to the central server at the end of the day.
There are 15 locations plus a central server. When location *A*
finishes its works, it sends its data to the central location. (When
it sends those data, it (*A*) updates a flag in the database
records. It can identify unsent records by those flag.)
Then the Oracle database in the central server should be updated
with location *A*�s records.
At that time the database in the central server would have been
updated with other locations data (for an example, location *B*�s
data). Then at the same time those data should go to location *A*
and location *A*�s database should be updated with those location
*B*�s data that the central server already have.
My question is how can I do this? What s the best technology to use?
(Web Services etc.) Can you recommend a book or some of web resources?
There is no one in the central server to do this. I hope the
location user to do this.
Still I am wondering which technology to use. I already experienced
in JSP, Java and Servlets. I have done some works with Xerces, Xalan
and WebServices.
If I am going to use Axis for this,
an I do this with Axis ?
Is it reliable Axis to do this? (Because this is a critical real
world application)
I appreciate if some of you give me some guidance here.
I would greatly appreciate your response.
Thanks and best regards,
Ranjith Kodikara
[EMAIL PROTECTED]
