Hi Patrick, at least I could do one half of the pending work yesternight...
On Aug 29, 2011, at 17:00 , Patrick Ohly wrote: > [...] > I've written such a test and found some problems: > 1. compile issue > 2. TPluginApiDS::apiAddItem() treated DB_DataMerged as a failure > and didn't return the local ID to > TCustomImplDS::implProcessItem(), which caused a segfault later > on (local ID NULL) > > Patches attached. Thanks. > But now I still see an unexpected Replace command. I looked up what's happening in the code and the replace is not unexpected to me: > Here's the processing > of the client's Add in a SyncEvolution server where that Add maps to an > item that was new for this client: > > [2011-08-29 16:43:57.474] adding "meeting on site, big meeting room" > [2011-08-29 16:43:57.479] InsertItemAsKey res=207 > [2011-08-29 16:43:57.480] Database adapter indicates that added item was > merged with pre-existing data (status 207) Status 207 is meant to indicate that the backend has added the item, but in the process merged the add with some additional data (from an external source, or another item in the sync set). So... ...the Server looks if the resulting item was already scheduled for sending to the client before (in its unmerged state): > [2011-08-29 16:43:57.480] TStdLogicDS::getConflictingItemByLocalID, found > RemoteID='1234567890!@#$%^&*()<>@dummy-rid', > LocalID='1234567890!@#$%^&*()<>@dummy-rid', syncop=add apparently yes... (so it seems to be an add/add conflict) > [2011-08-29 16:43:57.480] Preventing > localID='1234567890!@#$%^&*()<>@dummy-rid' to be sent to client > [2011-08-29 16:43:57.480] Item with > localID='1234567890!@#$%^&*()<>@dummy-rid' will NOT be sent to client > (slowsync match / duplicate prevention) ...sending this item is to be prevented, as we're not interested in sending the unmerged state. What we want to update the client with is the merged state, so next the server re-fetches the item from the backend, to get the merged state: > [2011-08-29 16:43:57.480] Created command 'Status' (outgoing) > [2011-08-29 16:43:57.484] ReadItemAsKey > aID=(1234567890!@#$%^&*()<>@dummy-rid,) res=0 > [2011-08-29 16:43:57.484] 'ScriptExecute' - Start executing Script, > name=afterreadscript [--][++] [->end] [->enclosing] > [...] > [2011-08-29 16:43:57.489] Forced conflict with corresponding item from server > DB So now we have the merged item (it is called forceConflict() for historical reasons, where fetching a particular item from the backend after loading the sync set was needed for creating a server response for an item that would not needed one). > [2011-08-29 16:43:57.489] Deleted command 'Status' (outgoing MsgID=0, CmdID=0) > [2011-08-29 16:43:57.489] ModifyMap called: aEntryType=normal, > aLocalID='1234567890!@#$%^&*()<>@dummy-rid, > aRemoteid='1234567890!@#$%^&*()<>@dummy-rid', aMapflags=0x0, aDelete=0 > [2011-08-29 16:43:57.490] - found no matching entry for localID > '1234567890!@#$%^&*()<>@dummy-rid' - creating new one, added=true > [2011-08-29 16:43:57.490] - Operation add performed (regular), Remote > ID=1234567890!@#$%^&*()<>@dummy-rid Local > ID=1234567890!@#$%^&*()<>@dummy-rid, status=201 > [2011-08-29 16:43:57.490] Deleted command 'Status' (outgoing MsgID=0, CmdID=0) Now the (new) client ID is mapped to the merge result's server ID. > [...] > > Later the server generates a Replace: > > –[2011-08-29 16:43:57.494] 'Item_Generate' - generating SyncML item, > SyncOp=wants-replace, RemoteID=1234567890!@#$%^&*()<>@dummy-rid [--][++] > [->end] [->enclosing] > +–[2011-08-29 16:43:57.494] 'ScriptExecute' - Start executing Script, > name=outgoingscript [--][++] [->end] [->enclosing] > [...] > +–[2011-08-29 16:43:57.505] 'issue' - issuing command, Cmd=Replace [--][++] > [->end] [->enclosing] > [2011-08-29 16:43:57.505] Command 'Replace': is 2-th counted cmd, > cmdsize(+tags needed to end msg)=500, available=38558 (maxfree=38558, > freeaftersend=38058, notUsableBufferBytes()=0) > [2011-08-29 16:43:57.505] Item remoteID='1234567890!@#$%^&*()<>@dummy-rid', > localID='', datasize=422 > [2011-08-29 16:43:57.506] Replace: issued as (outgoing MsgID=2, CmdID=5), now > queueing for status > [2011-08-29 16:43:57.506] Outgoing Message size is now 946 bytes > –[2011-08-29 16:43:57.506] End of 'issue' [->top] [->enclosing] > +–[2011-08-29 16:43:57.506] 'DSStateChange' - Datastore changes state, > datastore=calendar, oldstate=server_sync_gen_started, newstate=sync_gen_done > [--][++] [->end] [->enclosing] > –[2011-08-29 16:43:57.506] End of 'DSStateChange' [->top] [->enclosing] > [2011-08-29 16:43:57.506] engGenerateSyncCommands ended, > state='sync_gen_done', sync generation done > > Shouldn't this Item_Generate be suppressed in this case? No, that's the server updating the client with the result of the merge. Why would you expect this to be suppressed? On the contrary, this replace is explicitly forced by the 207 status handling code. Maybe we can sort out why you were expecting something else, before I implement the "other half" The other half: >>> So what's missing right now is a simple way to actually do the merge >>> in the backend, without re-implementing half of libsynthesis. The >>> (hopefully not so) longterm solution would be libvxxx. As a short term >>> workaround, which is not exactly super elegant but not dangerous >>> either, we could do the following: >>> >>> * we define another special return code for AddItem, say 419. >>> The backend can return it when it detects that the added data >>> SHOULD >>> be merged with what the backend already has, but can't do that >>> itself. >>> >>> * the engine would do the same things as (details see commit msg >>> in the patch below) for 207 but additionally it would: >>> * merge the incoming data with the data fetched from the backend >>> with the localID returned from AddItem() >>> * call UpdateItem() with the merge result in the backend >>> >>> Let me know what you think... >> >> That sounds like it would solve the problem. > > Is this something that you intend to work on or shall I give it a try > myself? I am confident I can do that tomorrow. Best Regards, Lukas Lukas Zeller, plan44.ch l...@plan44.ch - www.plan44.ch _______________________________________________ os-libsynthesis mailing list os-libsynthesis@synthesis.ch http://lists.synthesis.ch/mailman/listinfo/os-libsynthesis
