Re: Java class for ordering SQL statements in ODMG to avoid FK violations
Hi Jakob
On Thu, 18 Nov 2004 21:17:04 +0100, Jakob Braeuchi [EMAIL PROTECTED] wrote:
hi gerhard,
i applied a small fix in addCollectionEdges()
...
else
{
if (col instanceof ManageableCollection)
{
Collection newCol = new ArrayList();
CollectionUtils.addAll(newCol, ((ManageableCollection)
col).ojbIterator());
refObjects = newCol.toArray();
}
else if (col instanceof Collection)
{
refObjects = ((Collection) col).toArray();
}
else if (col instanceof Object[])
...
two of the reported problems were caused by the fact that ManageableCollection
does not extend Collection.
Oops... Yep, that's something I overlooked. I should have got suspicious when I
copied that error message from BrokerHelp.getCollectionIterator ... :-)
Gerhard
but i still have problems in org.apache.ojb.odmg.M2NTest.
Tests run: 207, Failures: 2, Errors: 6
jakob
Jakob Braeuchi schrieb:
hi gerhard,
thanks for your contribution. it looks really impressive !
i integrated your class into to current 1.0.x branch an ran all odmg
testcases without skipping known issues.
with the old reorder i get:
Tests run: 207, Failures: 2, Errors: 8
with your ordering i still have 6 errors:
Tests run: 207, Failures: 2, Errors: 6
ie this one:
3)
testStoreComplex(org.apache.ojb.odmg.M2NTest)org.apache.ojb.odmg.TransactionAbortedExceptionOJB
at
org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:174)
at
org.apache.ojb.odmg.TransactionImpl.doWriteObjects(TransactionImpl.java:324)
at
org.apache.ojb.odmg.TransactionImpl.prepare(TransactionImpl.java:624)
at org.apache.ojb.odmg.TransactionImpl.commit(TransactionImpl.java:581)
at org.apache.ojb.odmg.M2NTest.doTestStoreComplex(M2NTest.java:157)
at org.apache.ojb.odmg.M2NTest.testStoreComplex(M2NTest.java:137)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at org.apache.ojb.odmg.AllTests.main(AllTests.java:18)
Caused by: org.apache.ojb.broker.OJBRuntimeException: Given object
collection of type class
org.apache.ojb.odmg.M2NTest$MovieManageableCollection can not be managed
by OJB. Use Array, Collection or ManageableCollection instead!
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addCollectionEdges(ObjectEnvelopeOrdering.java:312)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addEdgesForVertex(ObjectEnvelopeOrdering.java:241)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.reorder(ObjectEnvelopeOrdering.java:133)
at
org.apache.ojb.odmg.ObjectEnvelopeTable.reorder(ObjectEnvelopeTable.java:588)
at
org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:148)
jakob
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Java class for ordering SQL statements in ODMG to avoid FK violations
Hi Jakob, On Fri, 19 Nov 2004 08:20:53 +0100 (MET), Jakob Braeuchi [EMAIL PROTECTED] wrote: hi armin, in the testcase i mentioned the auto-update is set to LINK. does the scenario you describe also work for odmg ? i was thinking of something similar in PB: the PB keeps track of the mn-implementors to be inserted and inserts them when the main objects are in the tables. If something like this could be realalized at PB level, it would remove the need for any ordering of m:n related objects. And this would in turn further reduce the probability of contradicting constraints. Note that of course deletions in the link table would have to occur before any deletion / update of the main objects. BTW, are the remaining failing test cases all in M2NTest and due to the bidirectional m:n relationship? I ran my tests against version 1.0.1 and got only the 2 failures in org.apache.ojb.odmg.CollectionsTest (testRemoveCollectionElementWithoutBackReference and testStoreDeleteCollectionElementWithBackReference) which also failed with the original ordering. Gerhard gerhards new class correctly orders the objects to be inserted, but cannot control the settings of auto-update. so the first obj inserted tries to update the indirection-table. jakob Jakob Braeuchi wrote: hi gerhard, the failing testcases in org.apache.ojb.odmg.M2NTest can not at all be fixed by reordering. the problem is the bidirectional m:n relationship with auto-update = LINK on both sides. no matter what side is inserted first pb tries to insert into the indirection-table as well. this will always fail. setting auto-update to OBJECT does also not help in this case because objects are inserted more than once which leads to a unique key violation. the only solution i see is to _defer_ the insertion of the indirection-table until all main objects are saved. unfortunately i have no idea how to do this :( hmm, if we change auto-update to NONE (instead false) for m:n relations, insert/update both sides, mark these objects with m:n relation for postprocessing and then populate the Indirection Table by using PBImpl#linkMtoN(Object obj, CollectionDescriptor cod, boolean insert). Note: I never tried this ;-) regards, Armin jakob Jakob Braeuchi schrieb: hi gerhard, thanks for your contribution. it looks really impressive ! i integrated your class into to current 1.0.x branch an ran all odmg testcases without skipping known issues. with the old reorder i get: Tests run: 207, Failures: 2, Errors: 8 with your ordering i still have 6 errors: Tests run: 207, Failures: 2, Errors: 6 ie this one: 3) testStoreComplex(org.apache.ojb.odmg.M2NTest)org.apache.ojb.odmg.TransactionAbortedExceptionOJB at org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:174) at org.apache.ojb.odmg.TransactionImpl.doWriteObjects(TransactionImpl.java:324) at org.apache.ojb.odmg.TransactionImpl.prepare(TransactionImpl.java:624) at org.apache.ojb.odmg.TransactionImpl.commit(TransactionImpl.java:581) at org.apache.ojb.odmg.M2NTest.doTestStoreComplex(M2NTest.java:157) at org.apache.ojb.odmg.M2NTest.testStoreComplex(M2NTest.java:137) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at org.apache.ojb.odmg.AllTests.main(AllTests.java:18) Caused by: org.apache.ojb.broker.OJBRuntimeException: Given object collection of type class org.apache.ojb.odmg.M2NTest$MovieManageableCollection can not be managed by OJB. Use Array, Collection or ManageableCollection instead! at org.apache.ojb.odmg.ObjectEnvelopeOrdering.addCollectionEdges(ObjectEnvelopeOrdering.java:312) at org.apache.ojb.odmg.ObjectEnvelopeOrdering.addEdgesForVertex(ObjectEnvelopeOrdering.java:241) at org.apache.ojb.odmg.ObjectEnvelopeOrdering.reorder(ObjectEnvelopeOrdering.java:133) at org.apache.ojb.odmg.ObjectEnvelopeTable.reorder(ObjectEnvelopeTable.java:588) at org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:148) jakob - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Java class for ordering SQL statements in ODMG to avoid FK violations
Hi Jakob, Jakob Braeuchi wrote: hi armin, in the testcase i mentioned the auto-update is set to LINK. does the scenario you describe also work for odmg ? suggestion only made for odmg i was thinking of something similar in PB: the PB keeps track of the mn-implementors to be inserted and inserts them when the main objects are in the tables. With PB-api this isn't possible I think, because with PB#store one object of the n-side is specified, you will never get all n-side objects at once. Current behavior is store n side object + all m-side objects and then insert indirection table entries - or I'm wrong? gerhards new class correctly orders the objects to be inserted, but cannot control the settings of auto-update. so the first obj inserted tries to update the indirection-table. That's why I suggest to set auto-update NONE for m:n relations in odmg. The main problem is to populate the m:n indirection table, currently we don't have the possibility to do this in a high-performance manner. For insert only we can use PB#addMtoNImplementor to populate indirection table, but this will fire many single sql-statements and will only work if we know which indirection entry we have to insert. An alternative way could be to use the link/unlink methods in BrokerHelper. An example using PB-api see ...broker.M2NTest#doTestStoreFFFX(Movie movie, String postfix) if we do something similar in an odmg-postprocess it should be possible to make m:n work. regards, Armin jakob Jakob Braeuchi wrote: hi gerhard, the failing testcases in org.apache.ojb.odmg.M2NTest can not at all be fixed by reordering. the problem is the bidirectional m:n relationship with auto-update = LINK on both sides. no matter what side is inserted first pb tries to insert into the indirection-table as well. this will always fail. setting auto-update to OBJECT does also not help in this case because objects are inserted more than once which leads to a unique key violation. the only solution i see is to _defer_ the insertion of the indirection-table until all main objects are saved. unfortunately i have no idea how to do this :( hmm, if we change auto-update to NONE (instead false) for m:n relations, insert/update both sides, mark these objects with m:n relation for postprocessing and then populate the Indirection Table by using PBImpl#linkMtoN(Object obj, CollectionDescriptor cod, boolean insert). Note: I never tried this ;-) regards, Armin jakob Jakob Braeuchi schrieb: hi gerhard, thanks for your contribution. it looks really impressive ! i integrated your class into to current 1.0.x branch an ran all odmg testcases without skipping known issues. with the old reorder i get: Tests run: 207, Failures: 2, Errors: 8 with your ordering i still have 6 errors: Tests run: 207, Failures: 2, Errors: 6 ie this one: 3) testStoreComplex(org.apache.ojb.odmg.M2NTest)org.apache.ojb.odmg.TransactionAbortedExceptionOJB at org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:174) at org.apache.ojb.odmg.TransactionImpl.doWriteObjects(TransactionImpl.java:324) at org.apache.ojb.odmg.TransactionImpl.prepare(TransactionImpl.java:624) at org.apache.ojb.odmg.TransactionImpl.commit(TransactionImpl.java:581) at org.apache.ojb.odmg.M2NTest.doTestStoreComplex(M2NTest.java:157) at org.apache.ojb.odmg.M2NTest.testStoreComplex(M2NTest.java:137) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at org.apache.ojb.odmg.AllTests.main(AllTests.java:18) Caused by: org.apache.ojb.broker.OJBRuntimeException: Given object collection of type class org.apache.ojb.odmg.M2NTest$MovieManageableCollection can not be managed by OJB. Use Array, Collection or ManageableCollection instead! at org.apache.ojb.odmg.ObjectEnvelopeOrdering.addCollectionEdges(ObjectEnvelopeOrdering.java:312) at org.apache.ojb.odmg.ObjectEnvelopeOrdering.addEdgesForVertex(ObjectEnvelopeOrdering.java:241) at org.apache.ojb.odmg.ObjectEnvelopeOrdering.reorder(ObjectEnvelopeOrdering.java:133) at org.apache.ojb.odmg.ObjectEnvelopeTable.reorder(ObjectEnvelopeTable.java:588) at org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:148) jakob Gerhard Grosse schrieb: Hi, for quite some time we were experiencing problems with FK constraint violations when committing ODMG transactions. We resolved these problems by inserting TransactionExt.flush() calls wherever necessary. However, this in turn produced deadlock problems under high server load. Therefore we decided to try and improve the ordering of the SQL sent to the database. These efforts resulted in a class org.ojb.odmg.ObjectEnvelopeOrdering which we hereby contribute to the OJB community, because other users seem to have similar problems.
Re: Java class for ordering SQL statements in ODMG to avoid FK violations
Dnia czwartek, 18 listopada 2004 14:54, Gerhard Grosse napisa: Hi, ... The ObjectEnvelopeOrdering class uses a graph-theoretical approach to order the object envelopes (see the JavaDoc for details). Edges in this graph are inserted only if there really is a need for ordering two object envelopes with respect to each other - determining this need is based on the modification state of both objects and the type of relation the two have to each other (1:1, 1:n, or m:n). This addresses issues (2)-(4). Issue (1) is not addressed as as niftily: Since we did not want to change the ObjectEnvelope class, we opted for a heuristic solution, where 'potential' relations between two objects are taken into account based on the class type of a reference. That sounds terrific! It woud be great to see it in OJB. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Java class for ordering SQL statements in ODMG to avoid FK violations
hi gerhard,
thanks for your contribution. it looks really impressive !
i integrated your class into to current 1.0.x branch an ran all odmg testcases
without skipping known issues.
with the old reorder i get:
Tests run: 207, Failures: 2, Errors: 8
with your ordering i still have 6 errors:
Tests run: 207, Failures: 2, Errors: 6
ie this one:
3)
testStoreComplex(org.apache.ojb.odmg.M2NTest)org.apache.ojb.odmg.TransactionAbortedExceptionOJB
at org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:174)
at org.apache.ojb.odmg.TransactionImpl.doWriteObjects(TransactionImpl.java:324)
at org.apache.ojb.odmg.TransactionImpl.prepare(TransactionImpl.java:624)
at org.apache.ojb.odmg.TransactionImpl.commit(TransactionImpl.java:581)
at org.apache.ojb.odmg.M2NTest.doTestStoreComplex(M2NTest.java:157)
at org.apache.ojb.odmg.M2NTest.testStoreComplex(M2NTest.java:137)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at org.apache.ojb.odmg.AllTests.main(AllTests.java:18)
Caused by: org.apache.ojb.broker.OJBRuntimeException: Given object collection of
type class org.apache.ojb.odmg.M2NTest$MovieManageableCollection can not be
managed by OJB. Use Array, Collection or ManageableCollection instead!
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addCollectionEdges(ObjectEnvelopeOrdering.java:312)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addEdgesForVertex(ObjectEnvelopeOrdering.java:241)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.reorder(ObjectEnvelopeOrdering.java:133)
at org.apache.ojb.odmg.ObjectEnvelopeTable.reorder(ObjectEnvelopeTable.java:588)
at org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:148)
jakob
Gerhard Grosse schrieb:
Hi,
for quite some time we were experiencing problems with FK constraint violations
when committing ODMG transactions. We resolved these problems by inserting
TransactionExt.flush() calls wherever necessary. However, this in turn produced
deadlock problems under high server load. Therefore we decided to try and
improve the ordering of the SQL sent to the database. These efforts resulted in
a class org.ojb.odmg.ObjectEnvelopeOrdering which we hereby contribute to the
OJB community, because other users seem to have similar problems.
To use this class, the private method reorder() in class
org.ojb.odmg.ObjectEnvelopeTable must be replaced with:
private void reorder()
{
if (needsCommit)
{
ObjectEnvelopeOrdering ordering =
new ObjectEnvelopeOrdering(transaction,mvOrderOfIds,mhtObjectEnvelopes);
ordering.reorder();
Identity[] newOrder = ordering.getOrdering();
mvOrderOfIds.clear();
for (int i = 0; i newOrder.length; i++)
{
mvOrderOfIds.add(newOrder[i]);
}
}
}
The ObjectEnvelopeOrdering class addresses the following problems we observed
with the original ordering algorithm:
(1) the ordering is based on the modified state of the objects only; therefore
if a reference in object A is set to null, OJB does not see that before
modification A referenced an object B to be deleted.
(2) 1:n and m:n collections are treated in the same way, although the
FK-location in the DB is quite different.
(3) the ordering algorithm is 'greedy' in the sense that once an object
modification has been put into the ordered list, it will stay at this position
even if later constraints are found that would require to move it to a position
further down.
(4) issue (3) is aggravated by the fact that the ordering does not take into
account the modification state of a referenced object. Therefore objects are
inserted into the ordered list even if there is no real need for ordering. Later
a real constraint me be encountered, but now the object is already frozen at its
new position.
The ObjectEnvelopeOrdering class uses a graph-theoretical approach to order the
object envelopes (see the JavaDoc for details). Edges in this graph are inserted
only if there really is a need for ordering two object envelopes with respect to
each other - determining this need is based on the modification state of both
objects and the type of relation the two have to each other (1:1, 1:n, or m:n).
This addresses issues (2)-(4).
Issue (1) is not addressed as as niftily: Since we did not want to change the
ObjectEnvelope class, we opted for a heuristic solution, where 'potential'
relations between two objects are taken into account based on the class type of
a reference.
Test results:
After using the ObjectEnvelopeOrdering class we were able to remove all flush
statements from our application without any test case failing. The results of
the JUnit tests of OJB did not change either (some fail before and after the
change), with one exception:
org.apache.ojb.broker.sequence.NativeIdentifierTest.testReferenceInsertUpdateODMG.
Before our change, this test
Re: Java class for ordering SQL statements in ODMG to avoid FK violations
hi gerhard,
i applied a small fix in addCollectionEdges()
...
else
{
if (col instanceof ManageableCollection)
{
Collection newCol = new ArrayList();
CollectionUtils.addAll(newCol, ((ManageableCollection)
col).ojbIterator());
refObjects = newCol.toArray();
}
else if (col instanceof Collection)
{
refObjects = ((Collection) col).toArray();
}
else if (col instanceof Object[])
...
two of the reported problems were caused by the fact that ManageableCollection
does not extend Collection.
but i still have problems in org.apache.ojb.odmg.M2NTest.
Tests run: 207, Failures: 2, Errors: 6
jakob
Jakob Braeuchi schrieb:
hi gerhard,
thanks for your contribution. it looks really impressive !
i integrated your class into to current 1.0.x branch an ran all odmg
testcases without skipping known issues.
with the old reorder i get:
Tests run: 207, Failures: 2, Errors: 8
with your ordering i still have 6 errors:
Tests run: 207, Failures: 2, Errors: 6
ie this one:
3)
testStoreComplex(org.apache.ojb.odmg.M2NTest)org.apache.ojb.odmg.TransactionAbortedExceptionOJB
at
org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:174)
at
org.apache.ojb.odmg.TransactionImpl.doWriteObjects(TransactionImpl.java:324)
at
org.apache.ojb.odmg.TransactionImpl.prepare(TransactionImpl.java:624)
at org.apache.ojb.odmg.TransactionImpl.commit(TransactionImpl.java:581)
at org.apache.ojb.odmg.M2NTest.doTestStoreComplex(M2NTest.java:157)
at org.apache.ojb.odmg.M2NTest.testStoreComplex(M2NTest.java:137)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at org.apache.ojb.odmg.AllTests.main(AllTests.java:18)
Caused by: org.apache.ojb.broker.OJBRuntimeException: Given object
collection of type class
org.apache.ojb.odmg.M2NTest$MovieManageableCollection can not be managed
by OJB. Use Array, Collection or ManageableCollection instead!
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addCollectionEdges(ObjectEnvelopeOrdering.java:312)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addEdgesForVertex(ObjectEnvelopeOrdering.java:241)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.reorder(ObjectEnvelopeOrdering.java:133)
at
org.apache.ojb.odmg.ObjectEnvelopeTable.reorder(ObjectEnvelopeTable.java:588)
at
org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:148)
jakob
Gerhard Grosse schrieb:
Hi,
for quite some time we were experiencing problems with FK constraint
violations
when committing ODMG transactions. We resolved these problems by
inserting
TransactionExt.flush() calls wherever necessary. However, this in turn
produced
deadlock problems under high server load. Therefore we decided to try and
improve the ordering of the SQL sent to the database. These efforts
resulted in
a class org.ojb.odmg.ObjectEnvelopeOrdering which we hereby contribute
to the
OJB community, because other users seem to have similar problems.
To use this class, the private method reorder() in class
org.ojb.odmg.ObjectEnvelopeTable must be replaced with:
private void reorder()
{
if (needsCommit)
{
ObjectEnvelopeOrdering ordering = new
ObjectEnvelopeOrdering(transaction,mvOrderOfIds,mhtObjectEnvelopes);
ordering.reorder();
Identity[] newOrder = ordering.getOrdering();
mvOrderOfIds.clear();
for (int i = 0; i newOrder.length; i++)
{
mvOrderOfIds.add(newOrder[i]);
}
}
}
The ObjectEnvelopeOrdering class addresses the following problems we
observed
with the original ordering algorithm:
(1) the ordering is based on the modified state of the objects only;
therefore
if a reference in object A is set to null, OJB does not see that before
modification A referenced an object B to be deleted.
(2) 1:n and m:n collections are treated in the same way, although the
FK-location in the DB is quite different.
(3) the ordering algorithm is 'greedy' in the sense that once an object
modification has been put into the ordered list, it will stay at this
position
even if later constraints are found that would require to move it to a
position
further down.
(4) issue (3) is aggravated by the fact that the ordering does not
take into
account the modification state of a referenced object. Therefore
objects are
inserted into the ordered list even if there is no real need for
ordering. Later
a real constraint me be encountered, but now the object is already
frozen at its
new position.
The ObjectEnvelopeOrdering class uses a graph-theoretical approach to
order the
object envelopes (see the JavaDoc for details). Edges in this graph
are inserted
only if there really is a need for
Re: Java class for ordering SQL statements in ODMG to avoid FK violations
hi gerhard,
the failing testcases in org.apache.ojb.odmg.M2NTest can not at all be fixed by
reordering.
the problem is the bidirectional m:n relationship with auto-update = LINK on
both sides. no matter what side is inserted first pb tries to insert into the
indirection-table as well. this will always fail.
setting auto-update to OBJECT does also not help in this case because objects
are inserted more than once which leads to a unique key violation.
the only solution i see is to _defer_ the insertion of the indirection-table
until all main objects are saved. unfortunately i have no idea how to do this :(
jakob
Jakob Braeuchi schrieb:
hi gerhard,
thanks for your contribution. it looks really impressive !
i integrated your class into to current 1.0.x branch an ran all odmg
testcases without skipping known issues.
with the old reorder i get:
Tests run: 207, Failures: 2, Errors: 8
with your ordering i still have 6 errors:
Tests run: 207, Failures: 2, Errors: 6
ie this one:
3)
testStoreComplex(org.apache.ojb.odmg.M2NTest)org.apache.ojb.odmg.TransactionAbortedExceptionOJB
at
org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:174)
at
org.apache.ojb.odmg.TransactionImpl.doWriteObjects(TransactionImpl.java:324)
at
org.apache.ojb.odmg.TransactionImpl.prepare(TransactionImpl.java:624)
at org.apache.ojb.odmg.TransactionImpl.commit(TransactionImpl.java:581)
at org.apache.ojb.odmg.M2NTest.doTestStoreComplex(M2NTest.java:157)
at org.apache.ojb.odmg.M2NTest.testStoreComplex(M2NTest.java:137)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at org.apache.ojb.odmg.AllTests.main(AllTests.java:18)
Caused by: org.apache.ojb.broker.OJBRuntimeException: Given object
collection of type class
org.apache.ojb.odmg.M2NTest$MovieManageableCollection can not be managed
by OJB. Use Array, Collection or ManageableCollection instead!
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addCollectionEdges(ObjectEnvelopeOrdering.java:312)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addEdgesForVertex(ObjectEnvelopeOrdering.java:241)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.reorder(ObjectEnvelopeOrdering.java:133)
at
org.apache.ojb.odmg.ObjectEnvelopeTable.reorder(ObjectEnvelopeTable.java:588)
at
org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:148)
jakob
Gerhard Grosse schrieb:
Hi,
for quite some time we were experiencing problems with FK constraint
violations
when committing ODMG transactions. We resolved these problems by
inserting
TransactionExt.flush() calls wherever necessary. However, this in turn
produced
deadlock problems under high server load. Therefore we decided to try and
improve the ordering of the SQL sent to the database. These efforts
resulted in
a class org.ojb.odmg.ObjectEnvelopeOrdering which we hereby contribute
to the
OJB community, because other users seem to have similar problems.
To use this class, the private method reorder() in class
org.ojb.odmg.ObjectEnvelopeTable must be replaced with:
private void reorder()
{
if (needsCommit)
{
ObjectEnvelopeOrdering ordering = new
ObjectEnvelopeOrdering(transaction,mvOrderOfIds,mhtObjectEnvelopes);
ordering.reorder();
Identity[] newOrder = ordering.getOrdering();
mvOrderOfIds.clear();
for (int i = 0; i newOrder.length; i++)
{
mvOrderOfIds.add(newOrder[i]);
}
}
}
The ObjectEnvelopeOrdering class addresses the following problems we
observed
with the original ordering algorithm:
(1) the ordering is based on the modified state of the objects only;
therefore
if a reference in object A is set to null, OJB does not see that before
modification A referenced an object B to be deleted.
(2) 1:n and m:n collections are treated in the same way, although the
FK-location in the DB is quite different.
(3) the ordering algorithm is 'greedy' in the sense that once an object
modification has been put into the ordered list, it will stay at this
position
even if later constraints are found that would require to move it to a
position
further down.
(4) issue (3) is aggravated by the fact that the ordering does not
take into
account the modification state of a referenced object. Therefore
objects are
inserted into the ordered list even if there is no real need for
ordering. Later
a real constraint me be encountered, but now the object is already
frozen at its
new position.
The ObjectEnvelopeOrdering class uses a graph-theoretical approach to
order the
object envelopes (see the JavaDoc for details). Edges in this graph
are inserted
only if there really is a need for ordering two object envelopes with
respect to
each other - determining this need is based on the modification state
of both
objects and
Re: Java class for ordering SQL statements in ODMG to avoid FK violations
Jakob Braeuchi wrote:
hi gerhard,
the failing testcases in org.apache.ojb.odmg.M2NTest can not at all be
fixed by reordering.
the problem is the bidirectional m:n relationship with auto-update =
LINK on both sides. no matter what side is inserted first pb tries to
insert into the indirection-table as well. this will always fail.
setting auto-update to OBJECT does also not help in this case because
objects are inserted more than once which leads to a unique key violation.
the only solution i see is to _defer_ the insertion of the
indirection-table until all main objects are saved. unfortunately i have
no idea how to do this :(
hmm, if we change auto-update to NONE (instead false) for m:n relations,
insert/update both sides, mark these objects with m:n relation for
postprocessing and then populate the Indirection Table by using
PBImpl#linkMtoN(Object obj, CollectionDescriptor cod, boolean insert).
Note: I never tried this ;-)
regards,
Armin
jakob
Jakob Braeuchi schrieb:
hi gerhard,
thanks for your contribution. it looks really impressive !
i integrated your class into to current 1.0.x branch an ran all odmg
testcases without skipping known issues.
with the old reorder i get:
Tests run: 207, Failures: 2, Errors: 8
with your ordering i still have 6 errors:
Tests run: 207, Failures: 2, Errors: 6
ie this one:
3)
testStoreComplex(org.apache.ojb.odmg.M2NTest)org.apache.ojb.odmg.TransactionAbortedExceptionOJB
at
org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:174)
at
org.apache.ojb.odmg.TransactionImpl.doWriteObjects(TransactionImpl.java:324)
at
org.apache.ojb.odmg.TransactionImpl.prepare(TransactionImpl.java:624)
at
org.apache.ojb.odmg.TransactionImpl.commit(TransactionImpl.java:581)
at org.apache.ojb.odmg.M2NTest.doTestStoreComplex(M2NTest.java:157)
at org.apache.ojb.odmg.M2NTest.testStoreComplex(M2NTest.java:137)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at org.apache.ojb.odmg.AllTests.main(AllTests.java:18)
Caused by: org.apache.ojb.broker.OJBRuntimeException: Given object
collection of type class
org.apache.ojb.odmg.M2NTest$MovieManageableCollection can not be
managed by OJB. Use Array, Collection or ManageableCollection instead!
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addCollectionEdges(ObjectEnvelopeOrdering.java:312)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addEdgesForVertex(ObjectEnvelopeOrdering.java:241)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.reorder(ObjectEnvelopeOrdering.java:133)
at
org.apache.ojb.odmg.ObjectEnvelopeTable.reorder(ObjectEnvelopeTable.java:588)
at
org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:148)
jakob
Gerhard Grosse schrieb:
Hi,
for quite some time we were experiencing problems with FK constraint
violations
when committing ODMG transactions. We resolved these problems by
inserting
TransactionExt.flush() calls wherever necessary. However, this in
turn produced
deadlock problems under high server load. Therefore we decided to try
and
improve the ordering of the SQL sent to the database. These efforts
resulted in
a class org.ojb.odmg.ObjectEnvelopeOrdering which we hereby
contribute to the
OJB community, because other users seem to have similar problems.
To use this class, the private method reorder() in class
org.ojb.odmg.ObjectEnvelopeTable must be replaced with:
private void reorder()
{
if (needsCommit)
{
ObjectEnvelopeOrdering ordering = new
ObjectEnvelopeOrdering(transaction,mvOrderOfIds,mhtObjectEnvelopes);
ordering.reorder();
Identity[] newOrder = ordering.getOrdering();
mvOrderOfIds.clear();
for (int i = 0; i newOrder.length; i++)
{
mvOrderOfIds.add(newOrder[i]);
}
}
}
The ObjectEnvelopeOrdering class addresses the following problems we
observed
with the original ordering algorithm:
(1) the ordering is based on the modified state of the objects only;
therefore
if a reference in object A is set to null, OJB does not see that before
modification A referenced an object B to be deleted.
(2) 1:n and m:n collections are treated in the same way, although the
FK-location in the DB is quite different.
(3) the ordering algorithm is 'greedy' in the sense that once an object
modification has been put into the ordered list, it will stay at this
position
even if later constraints are found that would require to move it to
a position
further down.
(4) issue (3) is aggravated by the fact that the ordering does not
take into
account the modification state of a referenced object. Therefore
objects are
inserted into the ordered list even if there is no real need for
ordering. Later
a real constraint me be encountered, but now the object is already
frozen at its
new
Re: Java class for ordering SQL statements in ODMG to avoid FK violations
hi armin,
in the testcase i mentioned the auto-update is set to LINK.
does the scenario you describe also work for odmg ? i was thinking of
something similar in PB: the PB keeps track of the mn-implementors to be
inserted and inserts them when the main objects are in the tables.
gerhards new class correctly orders the objects to be inserted, but cannot
control the settings of auto-update. so the first obj inserted tries to
update the indirection-table.
jakob
Jakob Braeuchi wrote:
hi gerhard,
the failing testcases in org.apache.ojb.odmg.M2NTest can not at all be
fixed by reordering.
the problem is the bidirectional m:n relationship with auto-update =
LINK on both sides. no matter what side is inserted first pb tries to
insert into the indirection-table as well. this will always fail.
setting auto-update to OBJECT does also not help in this case because
objects are inserted more than once which leads to a unique key
violation.
the only solution i see is to _defer_ the insertion of the
indirection-table until all main objects are saved. unfortunately i have
no idea how to do this :(
hmm, if we change auto-update to NONE (instead false) for m:n relations,
insert/update both sides, mark these objects with m:n relation for
postprocessing and then populate the Indirection Table by using
PBImpl#linkMtoN(Object obj, CollectionDescriptor cod, boolean insert).
Note: I never tried this ;-)
regards,
Armin
jakob
Jakob Braeuchi schrieb:
hi gerhard,
thanks for your contribution. it looks really impressive !
i integrated your class into to current 1.0.x branch an ran all odmg
testcases without skipping known issues.
with the old reorder i get:
Tests run: 207, Failures: 2, Errors: 8
with your ordering i still have 6 errors:
Tests run: 207, Failures: 2, Errors: 6
ie this one:
3)
testStoreComplex(org.apache.ojb.odmg.M2NTest)org.apache.ojb.odmg.TransactionAbortedExceptionOJB
at
org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:174)
at
org.apache.ojb.odmg.TransactionImpl.doWriteObjects(TransactionImpl.java:324)
at
org.apache.ojb.odmg.TransactionImpl.prepare(TransactionImpl.java:624)
at
org.apache.ojb.odmg.TransactionImpl.commit(TransactionImpl.java:581)
at org.apache.ojb.odmg.M2NTest.doTestStoreComplex(M2NTest.java:157)
at org.apache.ojb.odmg.M2NTest.testStoreComplex(M2NTest.java:137)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at org.apache.ojb.odmg.AllTests.main(AllTests.java:18)
Caused by: org.apache.ojb.broker.OJBRuntimeException: Given object
collection of type class
org.apache.ojb.odmg.M2NTest$MovieManageableCollection can not be
managed by OJB. Use Array, Collection or ManageableCollection instead!
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addCollectionEdges(ObjectEnvelopeOrdering.java:312)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.addEdgesForVertex(ObjectEnvelopeOrdering.java:241)
at
org.apache.ojb.odmg.ObjectEnvelopeOrdering.reorder(ObjectEnvelopeOrdering.java:133)
at
org.apache.ojb.odmg.ObjectEnvelopeTable.reorder(ObjectEnvelopeTable.java:588)
at
org.apache.ojb.odmg.ObjectEnvelopeTable.commit(ObjectEnvelopeTable.java:148)
jakob
Gerhard Grosse schrieb:
Hi,
for quite some time we were experiencing problems with FK constraint
violations
when committing ODMG transactions. We resolved these problems by
inserting
TransactionExt.flush() calls wherever necessary. However, this in
turn produced
deadlock problems under high server load. Therefore we decided to try
and
improve the ordering of the SQL sent to the database. These efforts
resulted in
a class org.ojb.odmg.ObjectEnvelopeOrdering which we hereby
contribute to the
OJB community, because other users seem to have similar problems.
To use this class, the private method reorder() in class
org.ojb.odmg.ObjectEnvelopeTable must be replaced with:
private void reorder()
{
if (needsCommit)
{
ObjectEnvelopeOrdering ordering = new
ObjectEnvelopeOrdering(transaction,mvOrderOfIds,mhtObjectEnvelopes);
ordering.reorder();
Identity[] newOrder = ordering.getOrdering();
mvOrderOfIds.clear();
for (int i = 0; i newOrder.length; i++)
{
mvOrderOfIds.add(newOrder[i]);
}
}
}
The ObjectEnvelopeOrdering class addresses the following problems we
observed
with the original ordering algorithm:
(1) the ordering is based on the modified state of the objects only;
therefore
if a reference in object A is set to null, OJB does not see that
before
