Hi John, You can set the task server queue to 1, which would effectively single-thread your CPF processes on that host (I think...).
But it seems strange that the http process you are calling out to would not be able to handle multiple requests. Are you sure you are sending a single document at a time to the web service? Maybe something in your code is sending multiple documents? At any rate, if setting the task server queue to 1 fixes this problem, that should help you figure out what the problem is (hopefully). You can also try testing the web service from outside MarkLogic by sending multiple requests to it. If you see the same issue there, then it might be that the service is not handling multiple requests the way you expect it to. -Danny -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of John Munson Sent: Wednesday, June 08, 2011 9:04 AM To: General MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] CPF and multithreading Danny, thanks very much for the info. My pipeline calls out to a 3rd-party text processing tool. When I submit each document to ML as a separate load, the 3rd-party tool marks up the text correctly, but when I submit multiple documents in one load, it only marks up the first one. I'm thinking maybe the tool isn't able to handle multithreaded requests, though I haven't been able to confirm that yet. In light of that, is there a config way to get CPF / the task server to stop multithreading and process each document sequentially? If not, maybe I'll have to write something to submit each document separately. Thanks again, John On 6/7/2011 7:46 PM, Danny Sokolsky wrote: > Hi John, > > CPF spawns tasks onto the task server, and the task server processes with > multiple threads. So that is normal to have interleaved log messages. > > So I guess the question is what is not working correctly when you have > multiple documents? > > 2 tips for debugging CPF apps are: > > 1) use the CPF trace event (which throws many log messages into the log) and > 2) look at the properties of documents after processing to see if any errors > occurred (any errors are logged into the properties). > > -Danny > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of John Munson > Sent: Tuesday, June 07, 2011 4:15 PM > To: [email protected] > Subject: [MarkLogic Dev General] CPF and multithreading > > Hi -- > > I'm very new to MarkLogic and to XQuery. Didn't see anything about this > in the archives... > > I've set up a Content Processing Framework pipeline using one of the > samples supplied in the installation. It works fine when I load a > single document, but not when I load more than one. I used xdmp:log() > to trace what's going on via the error log, and I was surprised to find > that my debug output was interleaved between the two documents I was > loading. This suggests that CPF handed the documents off to my > processing in separate threads. Does anyone know if that is how CPF is > designed to work? If so, then maybe the bad output I'm getting is due > to some other problem, but I'm wondering if it might be due to a > multithreading problem, given that it shows up only with multiple documents. > > Thanks, > John > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
