On Behalf Of Sam Mefford
Sent: August 22, 2017 16:18
To: MarkLogic Developer Discussion
Subject: [**EXTERNAL**] Re: [MarkLogic Dev General] Large job processing
question.
We generally write external applications for long-running jobs. Java is a
popular language for such jobs, and our Data Mov
o: MarkLogic Developer Discussion
mailto:general@developer.marklogic.com>>
Date: Tuesday, August 22, 2017 at 10:33 PM
To: MarkLogic Developer Discussion
mailto:general@developer.marklogic.com>>
Subject: Re: [MarkLogic Dev General] Large job processing question.
Is it smart enough not
r your cooperation.
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Ladner, Eric
(Eric.Ladner) [eric.lad...@chevron.com]
Sent: Tuesday, August 22, 2017 8:36 AM
To: general@developer.marklogic.com
Subject: [MarkLogic Dev General]
f Of Geert Josten
Sent: August 22, 2017 13:59
To: MarkLogic Developer Discussion
Subject: [**EXTERNAL**] Re: [MarkLogic Dev General] Large job processing
question.
Hi Eric,
Personally, I would probably let go of the all-docs-at-once approach, and spawn
processes for each input (sub)f
August 22, 2017 13:59
To: MarkLogic Developer Discussion
Subject: [**EXTERNAL**] Re: [MarkLogic Dev General] Large job processing
question.
Hi Eric,
Personally, I would probably let go of the all-docs-at-once approach, and spawn
processes for each input (sub)folder, and potentially for batc
vron.com>>
Reply-To: MarkLogic Developer Discussion
mailto:general@developer.marklogic.com>>
Date: Tuesday, August 22, 2017 at 4:36 PM
To: "general@developer.marklogic.com<mailto:general@developer.marklogic.com>"
mailto:general@developer.marklogic.com>>
Subject: [Ma
We have some large jobs (ingestion and validation of unstructured documents)
that have timeout issues.
The way the jobs are structured is structured is that the first job checks that
all the existing documents are valid (still exists on the file system). It
does this in two steps:
1) gath
