Re: [os-libsynthesis] temporary local ID + FinalizeLocalID
On Mon, 2013-06-10 at 12:35 +0200, Lukas Zeller wrote:
> On 10.06.2013, at 11:57, Patrick Ohly wrote:
> > I'll probably try something else: if commands were delayed in a
> > message which is marked Final, then force execution of the commands at
> > the end of processing that message and issue their Status commands, just
> > as syncing normally would.
>
> Sounds like a good compromise to me.
It seems to be working, too :-)
> > The effect will be that commands only get delayed across messages if
> > there will be more of them coming in anyway. The hope is that this will
> > avoid or at least minimize state change issues.
> >
> > Note that the problem that I am running into also exists in the
> > unmodified code: I expect similar issues when the server delays
> > processing because of timing.
>
> Probably not, because the server can only delay processing at the very
> beginning of the sync, while it loads the sync set in the background.
> So far, this was the only (but well tested with very long lasting sync
> set loads) case of delaying command execution. Delaying execution
> after sync has actually started is a new case.
Ah, I see. I had skipped over the
fLocalSyncDatastoreP->engIsStarted(false) check in the relevant code:
TSyncSession::processSyncOpItem()
...
// check if we can process it now
// Note: request time limit is active in server only.
if (!fLocalSyncDatastoreP->engIsStarted(false) || RemainingRequestTime()<0) {
aQueueForLater=true; // re-execute later...
return true; // ...but otherwise ok
}
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
___
os-libsynthesis mailing list
os-libsynthesis@synthesis.ch
http://lists.synthesis.ch/mailman/listinfo/os-libsynthesis
Re: [os-libsynthesis] temporary local ID + FinalizeLocalID
Hello Patrick, On 10.06.2013, at 11:57, Patrick Ohly wrote: >>> Looking at it like that, the question becomes: how does the server >>> decided that it still needs an answer from the client? >> >> Perhaps the problem is on the client side after all: in message msgID=2, >> the client client sent the Add command with a Final flag. In msgID=3, it >> acknowledges the server's SyncCmd with a Status, *without* a Final flag. >> The server therefore (in it's MessageEnded checks) continues with the >> session, whereas the client quits it. >> >> Should the client continue to set the Final flag as long as it is >> waiting for the server to finish? This is one of the things the specs did not nail and actual implementations varied. Some severely misbehaved when receiving extra (more than one per "phase"), and others were tolerant. Things are complicated further because is session scope, whereas the actual phase is datastore scope. This can lead to really strange overlaps when multiple datastores are involved. > It'd didn't do that because the server didn't set it either. > > My feeling is that untangling all these state changes is going to be > tricky. Undoubtedly. Untangling the code that hasn't become prettier over time would be one thing. However, the real problem is that there's no clear specs to even draw a state diagram in the first place. > I'll probably try something else: if commands were delayed in a > message which is marked Final, then force execution of the commands at > the end of processing that message and issue their Status commands, just > as syncing normally would. Sounds like a good compromise to me. > The effect will be that commands only get delayed across messages if > there will be more of them coming in anyway. The hope is that this will > avoid or at least minimize state change issues. > > Note that the problem that I am running into also exists in the > unmodified code: I expect similar issues when the server delays > processing because of timing. Probably not, because the server can only delay processing at the very beginning of the sync, while it loads the sync set in the background. So far, this was the only (but well tested with very long lasting sync set loads) case of delaying command execution. Delaying execution after sync has actually started is a new case. Best Regards, Lukas Zeller ___ os-libsynthesis mailing list os-libsynthesis@synthesis.ch http://lists.synthesis.ch/mailman/listinfo/os-libsynthesis
Re: [os-libsynthesis] temporary local ID + FinalizeLocalID
On Mon, 2013-06-10 at 11:45 +0200, Patrick Ohly wrote: > On Mon, 2013-06-10 at 11:27 +0200, Patrick Ohly wrote: > > On Sun, 2013-06-09 at 12:59 +0200, Lukas Zeller wrote: > > > Hello Patrick, > > > > > > On 07.06.2013, at 16:37, Patrick Ohly wrote: > > > > > > > Couldn't the ordering of statuses be enforced by queuing the response of > > > > commands which complete out of order, without forcing the strict > > > > ordering of command execution? > > > > > > I'd say yes - as long as the statuses arrive in the order the commands > > > were sent, the peer doesn't see in what order the commands were > > > actually executed. > > > > > > However, there are tons of edge cases to consider, especially for > > > split message commands () and multi-message issues in > > > general. The SyncML state machine was already a mess in the specs, and > > > got even more so by all the workarounds for peers that didn't work > > > correctly (or interpreted the weak specs differently) in one or the > > > other case. > > > > > > So while out of order execution and re-ordering statuses afterwards is > > > certainly thinkable, I don't have a firm idea how difficult that would > > > be to implement without breaking compatibility with all the weird > > > peers out there. My general feeling is: more complicated than it looks > > > on the first sight. > > > > My idea was to use the same delayed command queue for commands that > > finished and those that still need to execute: when encountering a > > finished command in that queue, the level processing the queue would > > issue the status command already prepared earlier. > > > > That way correct ordering would always be guaranteed. > > > > > > The server expected another message whereas the client closed > > > > the connection, leading to an error in SyncEvolution's message > > > > transport: > > > [...] > > > > > > > I am a bit at a loss stepping through the client and server states here > > > > that led to this situation. Lukas, can you help? > > > > > > I looked through the logs, but can't see right away what's going > > > wrong. I'd guess that the problem is the updating of the fNeedToAnswer > > > flag, which does not get properly set or reset due to the out-of-order > > > execution. > > > > What exactly does "fNeedToAnswer" mean? Initially I thought it meant "my > > peer needs to send me an answer", because you were talking about it in > > the context of the server sending a message and then expecting an answer > > that the client doesn't send. > > > > But after looking at the code more closely, it seems to have the meaning > > of "I need to send my answer". Right? > > > > In that case, the flag is irrelevant on the server except for (perhaps) > > some state changes, because the server always has to reply to the > > client. > > > > The problem in my case is not that the flag is set when it shouldn't be > > set (or vice versa), the problem is that the server expects another > > message. I think the client is correct in not sending that message. > > > > The key difference in the logs is that in the normal case, the server > > stays in the "map" phase, while in the failure case it goes > > > > > > Okay: > > > > [2013-06-07 15:55:13.697] ---> MessageEnded starts : old incoming > > state='sync', old outgoing state='sync', NeedToAnswer > > > > [2013-06-07 16:14:55.625] ---> MessageEnded finishes : new incoming > > state='map', new outgoing state='map', NeedToAnswer > > > > > > Failed: > > [2013-06-07 15:55:13.697] ---> MessageEnded starts : old incoming > > state='sync', old outgoing state='sync', NeedToAnswer > > > > [2013-06-07 15:55:13.697] ---> MessageEnded finishes : new incoming > > state='map', new outgoing state='sync', NeedToAnswer > > > > [2013-06-07 15:55:13.777] ---> MessageEnded starts : old incoming > > state='map', old outgoing state='sync', NeedToAnswer > > > > [2013-06-07 15:55:13.777] ---> MessageEnded finishes : new incoming > > state='map', new outgoing state='map', NeedToAnswer > > error> > > > > Looking at it like that, the question becomes: how does the server > > decided that it still needs an answer from the client? > > Perhaps the problem is on the client side after all: in message msgID=2, > the client client sent the Add command with a Final flag. In msgID=3, it > acknowledges the server's SyncCmd with a Status, *without* a Final flag. > The server therefore (in it's MessageEnded checks) continues with the > session, whereas the client quits it. > > Should the client continue to set the Final flag as long as it is > waiting for the server to finish? It'd didn't do that because the server didn't set it either. My feeling is that untangling all these state changes is going to be tricky. I'll probably try something else: if commands were delayed in a message which is marked Final, then force execution of the commands at the end of processing that message and issue their Status commands, just as syncing normally would. The eff
Re: [os-libsynthesis] temporary local ID + FinalizeLocalID
On Mon, 2013-06-10 at 11:27 +0200, Patrick Ohly wrote: > On Sun, 2013-06-09 at 12:59 +0200, Lukas Zeller wrote: > > Hello Patrick, > > > > On 07.06.2013, at 16:37, Patrick Ohly wrote: > > > > > Couldn't the ordering of statuses be enforced by queuing the response of > > > commands which complete out of order, without forcing the strict > > > ordering of command execution? > > > > I'd say yes - as long as the statuses arrive in the order the commands > > were sent, the peer doesn't see in what order the commands were > > actually executed. > > > > However, there are tons of edge cases to consider, especially for > > split message commands () and multi-message issues in > > general. The SyncML state machine was already a mess in the specs, and > > got even more so by all the workarounds for peers that didn't work > > correctly (or interpreted the weak specs differently) in one or the > > other case. > > > > So while out of order execution and re-ordering statuses afterwards is > > certainly thinkable, I don't have a firm idea how difficult that would > > be to implement without breaking compatibility with all the weird > > peers out there. My general feeling is: more complicated than it looks > > on the first sight. > > My idea was to use the same delayed command queue for commands that > finished and those that still need to execute: when encountering a > finished command in that queue, the level processing the queue would > issue the status command already prepared earlier. > > That way correct ordering would always be guaranteed. > > > > The server expected another message whereas the client closed > > > the connection, leading to an error in SyncEvolution's message > > > transport: > > [...] > > > > > I am a bit at a loss stepping through the client and server states here > > > that led to this situation. Lukas, can you help? > > > > I looked through the logs, but can't see right away what's going > > wrong. I'd guess that the problem is the updating of the fNeedToAnswer > > flag, which does not get properly set or reset due to the out-of-order > > execution. > > What exactly does "fNeedToAnswer" mean? Initially I thought it meant "my > peer needs to send me an answer", because you were talking about it in > the context of the server sending a message and then expecting an answer > that the client doesn't send. > > But after looking at the code more closely, it seems to have the meaning > of "I need to send my answer". Right? > > In that case, the flag is irrelevant on the server except for (perhaps) > some state changes, because the server always has to reply to the > client. > > The problem in my case is not that the flag is set when it shouldn't be > set (or vice versa), the problem is that the server expects another > message. I think the client is correct in not sending that message. > > The key difference in the logs is that in the normal case, the server > stays in the "map" phase, while in the failure case it goes > > > Okay: > > [2013-06-07 15:55:13.697] ---> MessageEnded starts : old incoming > state='sync', old outgoing state='sync', NeedToAnswer > > [2013-06-07 16:14:55.625] ---> MessageEnded finishes : new incoming > state='map', new outgoing state='map', NeedToAnswer > > > Failed: > [2013-06-07 15:55:13.697] ---> MessageEnded starts : old incoming > state='sync', old outgoing state='sync', NeedToAnswer > > [2013-06-07 15:55:13.697] ---> MessageEnded finishes : new incoming > state='map', new outgoing state='sync', NeedToAnswer > > [2013-06-07 15:55:13.777] ---> MessageEnded starts : old incoming > state='map', old outgoing state='sync', NeedToAnswer > > [2013-06-07 15:55:13.777] ---> MessageEnded finishes : new incoming > state='map', new outgoing state='map', NeedToAnswer > error> > > Looking at it like that, the question becomes: how does the server > decided that it still needs an answer from the client? Perhaps the problem is on the client side after all: in message msgID=2, the client client sent the Add command with a Final flag. In msgID=3, it acknowledges the server's SyncCmd with a Status, *without* a Final flag. The server therefore (in it's MessageEnded checks) continues with the session, whereas the client quits it. Should the client continue to set the Final flag as long as it is waiting for the server to finish? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ os-libsynthesis mailing list os-libsynthesis@synthesis.ch http://lists.synthesis.ch/mailman/listinfo/os-libsynthesis
Re: [os-libsynthesis] temporary local ID + FinalizeLocalID
On Sun, 2013-06-09 at 12:59 +0200, Lukas Zeller wrote: > Hello Patrick, > > On 07.06.2013, at 16:37, Patrick Ohly wrote: > > > Couldn't the ordering of statuses be enforced by queuing the response of > > commands which complete out of order, without forcing the strict > > ordering of command execution? > > I'd say yes - as long as the statuses arrive in the order the commands > were sent, the peer doesn't see in what order the commands were > actually executed. > > However, there are tons of edge cases to consider, especially for > split message commands () and multi-message issues in > general. The SyncML state machine was already a mess in the specs, and > got even more so by all the workarounds for peers that didn't work > correctly (or interpreted the weak specs differently) in one or the > other case. > > So while out of order execution and re-ordering statuses afterwards is > certainly thinkable, I don't have a firm idea how difficult that would > be to implement without breaking compatibility with all the weird > peers out there. My general feeling is: more complicated than it looks > on the first sight. My idea was to use the same delayed command queue for commands that finished and those that still need to execute: when encountering a finished command in that queue, the level processing the queue would issue the status command already prepared earlier. That way correct ordering would always be guaranteed. > > The server expected another message whereas the client closed > > the connection, leading to an error in SyncEvolution's message > > transport: > [...] > > > I am a bit at a loss stepping through the client and server states here > > that led to this situation. Lukas, can you help? > > I looked through the logs, but can't see right away what's going > wrong. I'd guess that the problem is the updating of the fNeedToAnswer > flag, which does not get properly set or reset due to the out-of-order > execution. What exactly does "fNeedToAnswer" mean? Initially I thought it meant "my peer needs to send me an answer", because you were talking about it in the context of the server sending a message and then expecting an answer that the client doesn't send. But after looking at the code more closely, it seems to have the meaning of "I need to send my answer". Right? In that case, the flag is irrelevant on the server except for (perhaps) some state changes, because the server always has to reply to the client. The problem in my case is not that the flag is set when it shouldn't be set (or vice versa), the problem is that the server expects another message. I think the client is correct in not sending that message. The key difference in the logs is that in the normal case, the server stays in the "map" phase, while in the failure case it goes Okay: [2013-06-07 15:55:13.697] ---> MessageEnded starts : old incoming state='sync', old outgoing state='sync', NeedToAnswer [2013-06-07 16:14:55.625] ---> MessageEnded finishes : new incoming state='map', new outgoing state='map', NeedToAnswer Failed: [2013-06-07 15:55:13.697] ---> MessageEnded starts : old incoming state='sync', old outgoing state='sync', NeedToAnswer [2013-06-07 15:55:13.697] ---> MessageEnded finishes : new incoming state='map', new outgoing state='sync', NeedToAnswer [2013-06-07 15:55:13.777] ---> MessageEnded starts : old incoming state='map', old outgoing state='sync', NeedToAnswer [2013-06-07 15:55:13.777] ---> MessageEnded finishes : new incoming state='map', new outgoing state='map', NeedToAnswer error> Looking at it like that, the question becomes: how does the server decided that it still needs an answer from the client? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ os-libsynthesis mailing list os-libsynthesis@synthesis.ch http://lists.synthesis.ch/mailman/listinfo/os-libsynthesis
