Hi Markus,
You may have a problem in your group communication setup if the
fragmentation layer is not working in JGroups. You may also have to
check your recovery log configuration to make sure that the columns
storing the SQL are large enough to contain the whole statement.
Otherwise there is no limit on the statement size in Sequoia, we do
insert Blobs of several MB so few KB should not be a problem.
Keep us posted with your progress,
Emmanuel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi Robert,
What group communications are you using? How were you able to deduce that
the problems were coming from group communications?
we are using JGroups (default with 2.10.10).
We asume it is the group communication, because when we have hanging
transactions we lookup the transaction ids and then do a "dump
transaction <id>" on both controllers.
On one of them (which got the initial statement) the whole transaction
statements are available, but on the second all parts - from the
statement including the blob to the end of the transaction - are missing.
Therefore we asume that it was the group communication. Also with only
one controller all does work well.
Thanks
Markus Wolf
Thanks, Robert
On 9/18/08 7:07 AM, "Markus Wolf" <[EMAIL PROTECTED]> wrote:
Hi Emmanuel,
we have had do reinitalize the whole cluster. To our luck this was only
a test environment and no production system.
After further investigating the problem we tracked it down to an error
in the controller group communication. We are uploading files in our
webapp which are afterwards stored as blobs in the database.
The whole process does work when only one controller is running, but if
the second controller is active only part of the transaction is
replicated to the second one and the transaction on both controller hangs.
The problem also only occurs if the file is larger than 2,7k (at least
in our tests). We also tested with a 27k file which result in hanging
transactions.
No exception is thrown from the controller or no log entry is done.
Is there an option to specify the file limits? Or maybe could it be
related to our hsqldb recovery log and blob handling?
Thanks for any help
Markus Wolf
we are using Sequoia 2.10.10 and have two controllers on dedicated
systems each with one backend database.
Yesterday I started a backup on controller2 and after that the database
was not enabled again. So I restarted the virtual-database but since
that the controller hangs.
After searching for some info about the problem (no log file entries) I
dicovered that there are 7 pending transactions on controller1 which
seem to block the controller. They do not seem to timeout and I have
found no option to rollback/timeout them with the console or JMX.
Yes, it is true that there is no option in the console to rollback a
transaction, this is probably a feature we should add. If you close the
client connections (can probably easily be done by resetting the app.
side connection pool) that will automatically rollback the connections.
Open transactions usually come from an application that does not
commit/rollback transactions in all cases (and thus transactions remain
uncommitted). Another caveat of the JDBC API is that a commit/rollback
automatically starts a new transaction. So if you don't force the
connection to setAutoCommit(true) after a commit/rollback they will be
in a new transaction. And Sequoia can only perform
enable.disable/backup/shutdown operations on transaction boundaries.
We never implemented a kill/abort command because it cannot be
implemented deterministically (what do you do is the transaction has
already committed on a backend and you try to cancel cluster-wide?).
This could however be implemented for open transaction with no active
request. Feel free to add a feature request in JIRA if you think this
could be a useful tool to have.
Am I missing something? Can someone give me a hint on how to stabilize
the system again. Or is the only solution to kill both controllers,
initialize the database again (or restore from dump) and synchronize the
controllers again afterwards?
In your case, the 2nd backend was still in backup mode waiting for
transactions to complete. If you are not able to terminate the
transactions by closing your application connections, you can reset the
virtual database by using a force shutdown (this will abort pending
transactions). You will probably have to resynchronize the 2nd
controller with its backend upon restart to make sure that they are
properly synchronized.
Hope this helps,
Emmanuel
--
Emmanuel Cecchet
FTO @ Frog Thinker
Open Source Development & Consulting
--
Web: http://www.frogthinker.org
email: [EMAIL PROTECTED]
Skype: emmanuel_cecchet
_______________________________________________
Sequoia mailing list
[email protected]
https://forge.continuent.org/mailman/listinfo/sequoia
