For ADD_CHANGE_DELETE, the contract for addSeedDocuments() basically says that you have to include *at least* the documents that were changed, added, or deleted since the previous stamp, and if no stamp is provided, it should return ALL specified documents. Are you doing that?
If you are, the next thing to look at is the computation of the version string. The version string is what is used to figure out if a change took place. You need this IN ADDITION TO the addSeedDocuments() doing the right thing. For deleted documents, obviously the processDocuments() should call the activities.deleteDocument() method. Does this sound like what your code is doing? Karl On Fri, May 24, 2019 at 4:25 PM Raman Gupta <rocketra...@gmail.com> wrote: > My team is creating a new repository connector. The source system has > a delta API that lets us know of all new, modified, and deleted > individual folders and documents since the last call to the API. Each > call to the delta API provides the changes, as well as a token which > can be provided on subsequent calls to get changes since that token > was generated/returned. > > What is the best approach to building a repo connector to a system > that has this type of delta API? > > Our first design was an implementation that specifies > `MODEL_ADD_CHANGE_DELETE` and then: > > * In addSeedDocuments, on the initial call we seed every document in > the source system. On subsequent calls, we use the delta API to seed > every added, modified, or deleted file. We return the delta API token > as the version value of addSeedDocuments, so that it an be used on > subsequent calls. > > * In processDocuments, we do the usual thing for each document identifier. > > On prototyping, this works for new docs, but "processDocuments" is > never triggered for modified and deleted docs. > > A second design we are considering is to use > MODEL_CHAINED_ADD_CHANGE_DELETE and have addSeedDocuments return only > one "virtual" document, which represents the root of the remote repo. > > Then, in "processDocuments" the new "document" is used to determine > all the child documents of that delta call, which are then added to > the queue via `activities.addDocumentReference`. To force the "virtual > seed" to trigger processDocuments again on the next call to > `addSeedDocuments`, we do `activities.deleteDocument(virtualDocId)` as > well. > > With this alternative design, the stage 1 seed effectively becomes a > no-op, and is just used as a mechanism to trigger stage 2. > > Thoughts? > > Regards, > Raman Gupta >
