Using ML 9: I have a process that quickly creates a large number of small documents, one for each item in a set of input items.
My code is basically: 1. Log that I’m about to act on the input item 2. Act on the input item (send the input item to a remote HTTP end point) 3. Create a new doc reflecting the input item I just acted on This code is within a try/catch and I log the exception, so I should know if there are any exceptions during this process by examining the log. I’m processing about 500K input items, with the processing spread over the 16 threads of my task server. So there are 16 tasks quickly writing these docs concurrently. I know the exact count of the input items and I get that count in the log, so I know that I’m actually processing all the items I should be. However, if I subsequently count the documents created in step 3 I’m short by about 1500, meaning that not all the docs got created, which should not be able to happen unless there was an exception between the log message and the document-insert() call, but I’m not finding any exceptions or other errors reported in the log. My question: is there anything that would cause docs to silently not get created under this kind of heavy-load? I would hope not but just wanted to make sure. I’m assuming this issue is my bug somewhere, but the code is pretty simple and I’m not seeing any obvious way the documents could not get created without a corresponding exception report. Thanks, Eliot -- Eliot Kimber http://contrext.com _______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
