Hello, Perhaps this is more a MongoDB issue, but maybe somebody has an idea:
The Camel MongoDB endpoint supports the persistent tail tracking feature (which means that it stores the value of an arbitrary increasing field in a document to reset the tailable cursor to this document when restarting/restting the cursor). The field can be for example a timestamp. Now my problem is: How do I get such an "constantly increasing field" if I haven't any? MongoDB hasn't something like an "auto increment" data type, so currently, I use a counters collection which generates sequences of increasing ids (as suggested in https://docs.mongodb.com/v3.0/tutorial/create-an-auto-incrementing-field/#auto-increment-counters-collection) However, this leads to some race conditions in a multi-threaded/multi-machine environment, as the process of "Sequence ID generation" and "insetion" is not atomic: - Consider two servers (A and B), every one inserts an update. A gets the first sequence number (e.g. 1), but B inserts it's document (with sequence number 2) earlier, so the natural (= insertion) order in the capped collection will now be "2 - 1". - Now, as the tailable cursor consumes the capped collections in natural order, it will start consuming the document with id 2. If the cursor is reset/regenerated at exactly this point, the persistent tail tracker will store "2" as the last processed document, although "1" was never processed, so 1 will be lost :(. Sounds academic, I know, but leads to some annoying errors in out application :( Thank you for any ideas. Best regards, Joerg -- View this message in context: http://camel.465427.n5.nabble.com/Mongodb-Persistent-tail-tracking-How-to-chose-an-increasing-field-for-tracking-tp5787124.html Sent from the Camel - Users mailing list archive at Nabble.com.
