Re: [Engine-devel] Global transaction in ovirt-engine
- Original Message - From: Michael Kublin mkub...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, February 12, 2013 4:59:19 PM Subject: Re: [Engine-devel] Global transaction in ovirt-engine - Original Message - From: Omer Frenkel ofren...@redhat.com To: Michael Kublin mkub...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, February 12, 2013 4:32:44 PM Subject: Re: [Engine-devel] Global transaction in ovirt-engine - Original Message - From: Michael Kublin mkub...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, February 12, 2013 11:38:33 AM Subject: [Engine-devel] Global transaction in ovirt-engine The issue is a following (I think I had raised it once), by default when any command is invoked in ovirt-engine a transaction is opened. Now, lets take a look on complicated command which is performing a lot of XML-RPC/HTTP calls and just simple update , such command is a main cause for the following problems: 1. Long transactions, each call increase a time of transaction 2. When transaction is opened a connection which is associated with it also in used, so a number of available DB connections reduced from connection pool. 3. A errors : transaction was cancelled and operation is failed on engine side, but successes at host side. 4. Performance - try to keep transactions as short as possible. 5. Performance - to many unneeded transactions 6. Bug debugging - it is very difficult to understand why something was committed and something not why transaction was reverted or not, when we have a lot of nested transactions. Now, what should be done by my opinion: 1. Change default behaviour - by default not to open a transaction (Today, transaction is opened and if we don't want a command should be marked by NonTransactiveCommandAttribute) - it is simple 2. The change described at 1 will require to rewrite and redesign some flows - more difficult, but also can be done pretty fast. By the way this can be or even should be done at any case. 3. All new commands should not be based on global transaction or keep in mind that global transaction can be disabled Benefits, 1. Better performance 2. Cleaner code 3. Less bugs 4. Increased scale Regards Michael ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel need to remember that transaction also gave us 10-min limit for command execution, not saying its good or bad, but disabling the transactions will allow commands to run forever, this 10 min limit actually, saved us few times when commands got stuck in all kind of locks etc.. Not exactly, if transaction has expired the JBOSS transaction framework (Arjuna) will not stop a running thread attached to such thread. We can configure it in such way that a running thread will be cancelled, but for now it is not. So current behavior is somewhat unexpected - it all determines if the thread will try to use the transaction after it was reaped off from him. Right? If this is the case - I think this is bad. Do we have a real requirement for commands to end their execution phase at 10 minutes? ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Global transaction in ovirt-engine
- Original Message - From: Michael Kublin mkub...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, February 12, 2013 11:38:33 AM Subject: [Engine-devel] Global transaction in ovirt-engine The issue is a following (I think I had raised it once), by default when any command is invoked in ovirt-engine a transaction is opened. Now, lets take a look on complicated command which is performing a lot of XML-RPC/HTTP calls and just simple update , such command is a main cause for the following problems: 1. Long transactions, each call increase a time of transaction 2. When transaction is opened a connection which is associated with it also in used, so a number of available DB connections reduced from connection pool. 3. A errors : transaction was cancelled and operation is failed on engine side, but successes at host side. 4. Performance - try to keep transactions as short as possible. 5. Performance - to many unneeded transactions 6. Bug debugging - it is very difficult to understand why something was committed and something not why transaction was reverted or not, when we have a lot of nested transactions. Now, what should be done by my opinion: 1. Change default behaviour - by default not to open a transaction (Today, transaction is opened and if we don't want a command should be marked by NonTransactiveCommandAttribute) - it is simple 2. The change described at 1 will require to rewrite and redesign some flows - more difficult, but also can be done pretty fast. By the way this can be or even should be done at any case. 3. All new commands should not be based on global transaction or keep in mind that global transaction can be disabled Benefits, 1. Better performance 2. Cleaner code 3. Less bugs 4. Increased scale Regards Michael ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel need to remember that transaction also gave us 10-min limit for command execution, not saying its good or bad, but disabling the transactions will allow commands to run forever, this 10 min limit actually, saved us few times when commands got stuck in all kind of locks etc.. Not exactly, if transaction has expired the JBOSS transaction framework (Arjuna) will not stop a running thread attached to such thread. We can configure it in such way that a running thread will be cancelled, but for now it is not. So current behavior is somewhat unexpected - it all determines if the thread will try to use the transaction after it was reaped off from him. Right? If this is the case - I think this is bad. Do we have a real requirement for commands to end their execution phase at 10 minutes? Propose patch for removing global transaction at endAction of CommandBase (this is a first step) http://gerrit.ovirt.org/#/c/12010/ ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Global transaction in ovirt-engine
The issue is a following (I think I had raised it once), by default when any command is invoked in ovirt-engine a transaction is opened. Now, lets take a look on complicated command which is performing a lot of XML-RPC/HTTP calls and just simple update , such command is a main cause for the following problems: 1. Long transactions, each call increase a time of transaction 2. When transaction is opened a connection which is associated with it also in used, so a number of available DB connections reduced from connection pool. 3. A errors : transaction was cancelled and operation is failed on engine side, but successes at host side. 4. Performance - try to keep transactions as short as possible. 5. Performance - to many unneeded transactions 6. Bug debugging - it is very difficult to understand why something was committed and something not why transaction was reverted or not, when we have a lot of nested transactions. Now, what should be done by my opinion: 1. Change default behaviour - by default not to open a transaction (Today, transaction is opened and if we don't want a command should be marked by NonTransactiveCommandAttribute) - it is simple 2. The change described at 1 will require to rewrite and redesign some flows - more difficult, but also can be done pretty fast. By the way this can be or even should be done at any case. 3. All new commands should not be based on global transaction or keep in mind that global transaction can be disabled Benefits, 1. Better performance 2. Cleaner code 3. Less bugs 4. Increased scale Regards Michael ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Global transaction in ovirt-engine
+1 it will also solve the bug of can do action checks running outside transaction. maybe we can also find a nice way to keep db only commands (without vdsm calls) to run within transaction to avoid repeat the code to open the transaction in such commands. by the way, we should solve the same for the end methods. - Original Message - From: Michael Kublin mkub...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, February 12, 2013 11:38:33 AM Subject: [Engine-devel] Global transaction in ovirt-engine The issue is a following (I think I had raised it once), by default when any command is invoked in ovirt-engine a transaction is opened. Now, lets take a look on complicated command which is performing a lot of XML-RPC/HTTP calls and just simple update , such command is a main cause for the following problems: 1. Long transactions, each call increase a time of transaction 2. When transaction is opened a connection which is associated with it also in used, so a number of available DB connections reduced from connection pool. 3. A errors : transaction was cancelled and operation is failed on engine side, but successes at host side. 4. Performance - try to keep transactions as short as possible. 5. Performance - to many unneeded transactions 6. Bug debugging - it is very difficult to understand why something was committed and something not why transaction was reverted or not, when we have a lot of nested transactions. Now, what should be done by my opinion: 1. Change default behaviour - by default not to open a transaction (Today, transaction is opened and if we don't want a command should be marked by NonTransactiveCommandAttribute) - it is simple 2. The change described at 1 will require to rewrite and redesign some flows - more difficult, but also can be done pretty fast. By the way this can be or even should be done at any case. 3. All new commands should not be based on global transaction or keep in mind that global transaction can be disabled Benefits, 1. Better performance 2. Cleaner code 3. Less bugs 4. Increased scale Regards Michael ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Global transaction in ovirt-engine
it will also solve the bug of can do action checks running outside - when not executing it from opened transaction ofcourse :-), but now it happens also by mistake. - Original Message - From: Liron Aravot lara...@redhat.com To: Michael Kublin mkub...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, February 12, 2013 1:06:52 PM Subject: Re: [Engine-devel] Global transaction in ovirt-engine +1 it will also solve the bug of can do action checks running outside transaction. maybe we can also find a nice way to keep db only commands (without vdsm calls) to run within transaction to avoid repeat the code to open the transaction in such commands. by the way, we should solve the same for the end methods. - Original Message - From: Michael Kublin mkub...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, February 12, 2013 11:38:33 AM Subject: [Engine-devel] Global transaction in ovirt-engine The issue is a following (I think I had raised it once), by default when any command is invoked in ovirt-engine a transaction is opened. Now, lets take a look on complicated command which is performing a lot of XML-RPC/HTTP calls and just simple update , such command is a main cause for the following problems: 1. Long transactions, each call increase a time of transaction 2. When transaction is opened a connection which is associated with it also in used, so a number of available DB connections reduced from connection pool. 3. A errors : transaction was cancelled and operation is failed on engine side, but successes at host side. 4. Performance - try to keep transactions as short as possible. 5. Performance - to many unneeded transactions 6. Bug debugging - it is very difficult to understand why something was committed and something not why transaction was reverted or not, when we have a lot of nested transactions. Now, what should be done by my opinion: 1. Change default behaviour - by default not to open a transaction (Today, transaction is opened and if we don't want a command should be marked by NonTransactiveCommandAttribute) - it is simple 2. The change described at 1 will require to rewrite and redesign some flows - more difficult, but also can be done pretty fast. By the way this can be or even should be done at any case. 3. All new commands should not be based on global transaction or keep in mind that global transaction can be disabled Benefits, 1. Better performance 2. Cleaner code 3. Less bugs 4. Increased scale Regards Michael ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Global transaction in ovirt-engine
- Original Message - From: Michael Kublin mkub...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, February 12, 2013 11:38:33 AM Subject: [Engine-devel] Global transaction in ovirt-engine The issue is a following (I think I had raised it once), by default when any command is invoked in ovirt-engine a transaction is opened. Now, lets take a look on complicated command which is performing a lot of XML-RPC/HTTP calls and just simple update , such command is a main cause for the following problems: 1. Long transactions, each call increase a time of transaction 2. When transaction is opened a connection which is associated with it also in used, so a number of available DB connections reduced from connection pool. 3. A errors : transaction was cancelled and operation is failed on engine side, but successes at host side. 4. Performance - try to keep transactions as short as possible. 5. Performance - to many unneeded transactions 6. Bug debugging - it is very difficult to understand why something was committed and something not why transaction was reverted or not, when we have a lot of nested transactions. Now, what should be done by my opinion: 1. Change default behaviour - by default not to open a transaction (Today, transaction is opened and if we don't want a command should be marked by NonTransactiveCommandAttribute) - it is simple 2. The change described at 1 will require to rewrite and redesign some flows - more difficult, but also can be done pretty fast. By the way this can be or even should be done at any case. 3. All new commands should not be based on global transaction or keep in mind that global transaction can be disabled Benefits, 1. Better performance 2. Cleaner code 3. Less bugs 4. Increased scale Regards Michael ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel need to remember that transaction also gave us 10-min limit for command execution, not saying its good or bad, but disabling the transactions will allow commands to run forever, this 10 min limit actually, saved us few times when commands got stuck in all kind of locks etc.. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Global transaction in ovirt-engine
- Original Message - From: Omer Frenkel ofren...@redhat.com To: Michael Kublin mkub...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, February 12, 2013 4:32:44 PM Subject: Re: [Engine-devel] Global transaction in ovirt-engine - Original Message - From: Michael Kublin mkub...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, February 12, 2013 11:38:33 AM Subject: [Engine-devel] Global transaction in ovirt-engine The issue is a following (I think I had raised it once), by default when any command is invoked in ovirt-engine a transaction is opened. Now, lets take a look on complicated command which is performing a lot of XML-RPC/HTTP calls and just simple update , such command is a main cause for the following problems: 1. Long transactions, each call increase a time of transaction 2. When transaction is opened a connection which is associated with it also in used, so a number of available DB connections reduced from connection pool. 3. A errors : transaction was cancelled and operation is failed on engine side, but successes at host side. 4. Performance - try to keep transactions as short as possible. 5. Performance - to many unneeded transactions 6. Bug debugging - it is very difficult to understand why something was committed and something not why transaction was reverted or not, when we have a lot of nested transactions. Now, what should be done by my opinion: 1. Change default behaviour - by default not to open a transaction (Today, transaction is opened and if we don't want a command should be marked by NonTransactiveCommandAttribute) - it is simple 2. The change described at 1 will require to rewrite and redesign some flows - more difficult, but also can be done pretty fast. By the way this can be or even should be done at any case. 3. All new commands should not be based on global transaction or keep in mind that global transaction can be disabled Benefits, 1. Better performance 2. Cleaner code 3. Less bugs 4. Increased scale Regards Michael ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel need to remember that transaction also gave us 10-min limit for command execution, not saying its good or bad, but disabling the transactions will allow commands to run forever, this 10 min limit actually, saved us few times when commands got stuck in all kind of locks etc.. Not exactly, if transaction has expired the JBOSS transaction framework (Arjuna) will not stop a running thread attached to such thread. We can configure it in such way that a running thread will be cancelled, but for now it is not. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
