Hi Justin,
Here is the actual code that returns XDMP-CONFLICTINGUPDATES Error:.
---
declareUpdate();
var mainDataResponse =
rawColls.documents()
.where(rawColls.byExample({personId: id}))
.result('iterator');
for (var result of mainDataResponse.results) {
var normalizedViewObj = cts.doc(matViewUri).toObject();
var matOriginalDoc = cts.doc(matViewUri);
var dateNow =new Date().toUTCString();
// if (doc.firstname !=null)
normalizedViewObj.firstName = result.document.firstname;
// if (doc.lastName !=null)
normalizedViewObj.lastName = result.document.lastname;
// if (doc.dateOfBirth !=null)
normalizedViewObj.dateOfBirth = result.document.dob;
xdmp.nodeReplace(cts.doc(matViewUri),normalizedViewObj),
normalizedViewObj = null;
}
---
On Thu, Sep 29, 2016 at 3:00 PM, <general-requ...@developer.marklogic.com>
wrote:
> Send General mailing list submissions to
> general@developer.marklogic.com
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://developer.marklogic.com/mailman/listinfo/general
> or, via email, send a message with subject or body 'help' to
> general-requ...@developer.marklogic.com
>
> You can reach the person managing the list at
> general-ow...@developer.marklogic.com
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of General digest..."
>
>
> Today's Topics:
>
> 1. Re: General Digest, Vol 147, Issue 51 (Justin Makeig)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 29 Sep 2016 16:32:44 +0000
> From: Justin Makeig <justin.mak...@marklogic.com>
> Subject: Re: [MarkLogic Dev General] General Digest, Vol 147, Issue 51
> To: Shiv Shankar <shiv.shivshan...@gmail.com>
> Cc: "general@developer.marklogic.com"
> <general@developer.marklogic.com>
> Message-ID: <22fded82-fc67-4aad-b1f9-2eecb5b6a...@marklogic.com>
> Content-Type: text/plain; charset="us-ascii"
>
> Can you show your actual code?
>
> Justin
>
>
> --
> Justin Makeig
> Director, Product Management
> MarkLogic
> jmak...@marklogic.com
>
>
> > On Sep 29, 2016, at 7:09 AM, Shiv Shankar <shiv.shivshan...@gmail.com>
> wrote:
> >
> > HI Justin,
> > Thanks for your quick reply. However, I used both
> declareUpdate({explicitCommit: true}); and declareUpdate(); Still same
> issue.
> >
> > I am just reading from one collection and updating into another document
> in a different collection. I managed with insertDocument, but this over
> writes the docs which I don't want. I just want to update the attributes (
> it is like UPDATE SET field1:=value, field2=value in loop in oracle world)
> >
> > Note: I used lock/unlock options, but getting no lock on that target
> record. I used isolation level as part of eval. Couldn't get through it.
> > XDMP-CONFLICTINGUPDATES: xdmp.eval("//{explicitCommit:
> true}\ndeclareUpdate(); \nvar json = \"/Mark...", {}, <options
> xmlns="xdmp:eval"><database>67298589469423423423</database><root>/</root><isolation...</options>)
> --Conflicting updates xdmp.nodeReplace(cts.doc("
> >
> >
> > Any thoughts?
> >
> > On Thu, Sep 29, 2016 at 7:55 AM, <general-request@developer.
> marklogic.com> wrote:
> > Send General mailing list submissions to
> > general@developer.marklogic.com
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > http://developer.marklogic.com/mailman/listinfo/general
> > or, via email, send a message with subject or body 'help' to
> > general-requ...@developer.marklogic.com
> >
> > You can reach the person managing the list at
> > general-ow...@developer.marklogic.com
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of General digest..."
> >
> >
> > Today's Topics:
> >
> > 1. Re: XDMP-CONFLICTINGUPDATES: while using xdmp.nodeReplace
> > (Justin Makeig)
> > 2. Re: mlcp Transaction Errors - SVC-EXTIME and XDMP-NOTXN #CGO#
> > (Jain, Abhishek)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Wed, 28 Sep 2016 21:02:26 +0000
> > From: Justin Makeig <justin.mak...@marklogic.com>
> > Subject: Re: [MarkLogic Dev General] XDMP-CONFLICTINGUPDATES: while
> > using xdmp.nodeReplace
> > To: MarkLogic Developer Discussion <general@developer.marklogic.com>
> > Message-ID: <4136b8ca-113a-4373-b6d3-e54d34f15...@marklogic.com>
> > Content-Type: text/plain; charset="us-ascii"
> >
> > The general pattern for doing updates on persisted JSON documents in
> JavaScript is to
> > 1) get the document
> > 2) turn it into an object
> > 3) update the object
> > 4) replace the document node with the updated object
> >
> > Here's an example that updates the `balance` property of every document
> in the `accounts` collection:
> >
> > declareUpdate();
> > for(var doc of fn.collection('accounts')) {
> > var account = doc.toObject();
> > account.balance = account.balance * 1.05;
> > xdmp.nodeReplace(doc, account);
> > }
> >
> > Note that you also need a declareUpdate(). The transaction is committed
> (or rolledback) automatically, so you don't need the xdmp.commit().
> >
> > Justin
> >
> > --
> > Justin Makeig
> > Director, Product Management
> > MarkLogic
> > jmak...@marklogic.com
> >
> > > On Sep 28, 2016, at 1:46 PM, Shiv Shankar <shiv.shivshan...@gmail.com>
> wrote:
> > >
> > > Hi Rob,
> > > Thanks for quick reply,
> > >
> > > Here is the sample scenario.
> > >
> > > I would be building random data and put in to newObjData in a for loop.
> > >
> > > 01.json = {"firstName": "abc", "lastName": "xyz"}
> > >
> > > newObjectData.firstName= "Donald";
> > >
> > > for Loop
> > > {
> > > xdmp.nodeReplace(cts.doc("/test/01.json"),newObjectData);
> > > xdmp.commit();
> > > }
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Sep 28, 2016 at 4:37 PM, Rob Szkutak <
> rob.szku...@marklogic.com> wrote:
> > > Hi,
> > >
> > > Do you have a reproducible case of this I could look at?
> > >
> > > Best,
> > > Rob
> > >
> > > Rob Szkutak
> > > Senior Consultant
> > > MarkLogic Corporation
> > > rob.szku...@marklogic.com
> > > www.marklogic.com
> > >
> > > From: Shiv Shankar [shiv.shivshan...@gmail.com]
> > > Sent: Wednesday, September 28, 2016 3:35 PM
> > > To: general@developer.marklogic.com; Rob Szkutak
> > > Subject: XDMP-CONFLICTINGUPDATES: while using xdmp.nodeReplace
> > >
> > > Hi Rob,
> > >
> > > I am getting XDMP-CONFLICTINGUPDATES: while using xdmp.nodeReplace to
> update JSON attributes in FOR loop, instead of over-writing entire
> document. Any advice.
> > >
> > > BTW, I used xdmp.eval, but it is not identifying the java script
> variables.
> > >
> > > Thanks
> > > ShivShankar.
> > >
> > > _______________________________________________
> > > General mailing list
> > > General@developer.marklogic.com
> > > Manage your subscription at:
> > > http://developer.marklogic.com/mailman/listinfo/general
> >
> >
> >
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Thu, 29 Sep 2016 11:54:54 +0000
> > From: "Jain, Abhishek" <abhishek.b.j...@capgemini.com>
> > Subject: Re: [MarkLogic Dev General] mlcp Transaction Errors -
> > SVC-EXTIME and XDMP-NOTXN #CGO#
> > To: MarkLogic Developer Discussion <general@developer.marklogic.com>
> > Message-ID:
> > <5A37843F322E824AA0D8359ED88CD01F276FF29D@IN-HYD-DAGND2.
> corp.capgemini.com>
> >
> > Content-Type: text/plain; charset="utf-8"
> >
> > Great , Expected !! As it was pretty optimized MLCP Query.
> > BTW, addressing the second part of your question about logging
> errors/failed files ?
> > I?m also working on similar stuff, this can be done in three ways ?
> >
> >
> > 1. Modifying MLCP library to write all the failed IDs and
> corresponding files in a separate log file
> >
> > However that requires additional work(java program or x-query ) program
> to create new input files of failed record.(Tedious)
> >
> > 2. Second way, we need to write a pre-commit trigger, and attach
> to database. In pre-commit action module we can
> >
> > Write custom logs.
> >
> > 3. Similar module can also be written for MLCP Transform.
> >
> >
> >
> > All three approaches have limitations , which one to choose depends on
> what exactly we want to log.
> >
> > In my case it pre-commit solution does well.
> >
> > I wonder MLCP doesn?t provide RE-TRY option.
> >
> > Thanks and Regards,
> > [Email_CBE.gif]Abhishek Jain
> > Associate Consultant
> >
> > From: general-boun...@developer.marklogic.com [mailto:general-bounces@
> developer.marklogic.com] On Behalf Of Stuart Myles
> > Sent: Friday, September 23, 2016 8:45 PM
> > To: MarkLogic Developer Discussion
> > Subject: Re: [MarkLogic Dev General] mlcp Transaction Errors -
> SVC-EXTIME and XDMP-NOTXN #CGO#
> >
> > Thanks! This helped me prevent the errors from occurring and - as a
> bonus - significantly sped up my ingestion.
> >
> > I couldn't use exactly the mlcp command line you suggested, since - in
> the version of mlcp I'm using - -input_file_type xml isn't allowed, I had
> to use -input_file_type documents instead. Also, my input files don't need
> to be split. However, bumping up the threads used (to 30 in my case) made
> the transaction / timeout complaints go away. And now I'm ingesting 100,000
> documents in 12 minutes, rather than one hour. Much better!
> >
> > Regards,
> >
> > Stuart
> >
> >
> >
> > On Fri, Sep 23, 2016 at 3:34 AM, Jain, Abhishek <
> abhishek.b.j...@capgemini.com<mailto:abhishek.b.j...@capgemini.com>>
> wrote:
> > Hi Stuart,
> >
> > MLCP comes with various options, and can be used in various
> combinations depending on the file size, memory available and
> > Other number of nodes, forest etc.
> >
> > If you want to try a quick solution you can try this mlcp command :
> > mlcp import -host yourhost -port 8000 -username userName -password
> PASSWORD -input_file_type xml -input_file_path TempData -thread_count
> -thread_count_per_split 3 -batch_size 200 -transaction_size 20
> -max_split_size 33554432 -split_input true
> > change username, input file type etc accordingly.
> > It?s always good to use splits and threads when working with huge
> dataset.
> > Some performance matrix you can consider while using above mlcp :
> >
> > 1. In app server settings you can check if connection time out is
> set to 0.
> >
> > 2. Default spilt size is 32MB, if you can change -max_split_size
> 33554432 ( it take in bytes, if your file is bigger )
> >
> > 3. Make sure split and thread ratio remains 1:2 or 1:3 for example
> > If your document size is 10 MB, and your split size is 1000,000 (1 MB)
> then 10/1 = 10 splits
> > Then you should create 20 or 30 thread for best CPU utilization.
> >
> > 4. The above mlcp does well with 150 Million rows, should work for
> you as well.
> >
> > 5. I assume you have a nice good RAM > 4GB alteast.
> >
> > Thanks and Regards,
> > [Email_CBE.gif]Abhishek Jain
> > Associate Consultant
> > Capgemini India | Hyderabad
> >
> > From: general-boun...@developer.marklogic.com<mailto:general-
> boun...@developer.marklogic.com> [mailto:general-bounces@
> developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>]
> On Behalf Of Stuart Myles
> > Sent: Thursday, September 22, 2016 11:52 PM
> > To: MarkLogic Developer Discussion
> > Subject: [MarkLogic Dev General] mlcp Transaction Errors - SVC-EXTIME
> and XDMP-NOTXN
> >
> > When I'm loading directories of slightly fewer than 100,000 XML files
> into a large MarkLogic instance, I often get timeout and transaction
> errors. If I re-run the same directory of files which got those errors, I
> typically don't get any errors.
> >
> > So, I have a few questions:
> >
> > * Can I get prevent the errors from happening in the first place - e.g.
> by tuning MarkLogic parameters or altering my use of mlcp?
> > * If I do get errors, what is the best way to get a report on the files
> which failed, so I can retry just those ones? Is the best option for me to
> write some code to pick out the errors from the log file? And, if so, am I
> guaranteed to get all of the files reported?
> >
> > Some Details
> >
> > The command line template is
> >
> > mlcp.sh import -username {1} -password {2} -host localhost -port {4}
> -input_file_path {5} -output_uri_replace \"{6},'{7}'\"
> >
> > Sometimes, the imports run just fine. However, often I get a large
> number of SVC-EXTIME errors followed by a XDMP-NOTXN error. For example:
> >
> > 16/09/22 17:54:03 ERROR mapreduce.ContentWriter: SVC-EXTIME: Time limit
> exceeded
> > 16/09/22 17:54:03 WARN mapreduce.ContentWriter: Failed document
> 029ccd8ac3323658277ca28fead7a73d.0.xml in file:/mnt/ingestion/
> MarkLogicIngestion/smyles/todo/2014_0005.done/
> 029ccd8ac3323658277ca28fead7a73d.0.xml
> > 16/09/22 17:54:03 ERROR mapreduce.ContentWriter: SVC-EXTIME: Time limit
> exceeded
> > 16/09/22 17:54:03 WARN mapreduce.ContentWriter: Failed document
> 02eb4562784255e249c4ec3ed472f9aa.1.xml in file:/mnt/ingestion/
> MarkLogicIngestion/smyles/todo/2014_0005.done/
> 02eb4562784255e249c4ec3ed472f9aa.1.xml
> > 16/09/22 17:54:04 INFO contentpump.LocalJobRunner: completed 33%
> > 16/09/22 17:54:21 ERROR mapreduce.ContentWriter: XDMP-NOTXN: No
> transaction with identifier 9076269665213828952
> >
> > So far, I'm just rerunning the entire directory again. Most of the time,
> it ingests fine on the second attempt. However, I have thousands of these
> directories to process. So, I would prefer to avoid getting the errors in
> the first place. Failing that, I would like to capture the errors and just
> retry the files which failed.
> >
> > Any help much appreciated.
> >
> > Regards,
> >
> > Stuart
> >
> >
> >
> > This message contains information that may be privileged or confidential
> and is the property of the Capgemini Group. It is intended only for the
> person to whom it is addressed. If you are not the intended recipient, you
> are not authorized to read, print, retain, copy, disseminate, distribute,
> or use this message or any part thereof. If you receive this message in
> error, please notify the sender immediately and delete all copies of this
> message.
> >
> > _______________________________________________
> > General mailing list
> > General@developer.marklogic.com<mailto:General@developer.marklogic.com>
> > Manage your subscription at:
> > http://developer.marklogic.com/mailman/listinfo/general
> >
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: http://developer.marklogic.com/pipermail/general/
> attachments/20160929/e5f311a1/attachment.html
> > -------------- next part --------------
> > A non-text attachment was scrubbed...
> > Name: image001.gif
> > Type: image/gif
> > Size: 1616 bytes
> > Desc: image001.gif
> > Url : http://developer.marklogic.com/pipermail/general/
> attachments/20160929/e5f311a1/attachment.gif
> >
> > ------------------------------
> >
> > _______________________________________________
> > General mailing list
> > General@developer.marklogic.com
> > Manage your subscription at:
> > http://developer.marklogic.com/mailman/listinfo/general
> >
> >
> > End of General Digest, Vol 147, Issue 51
> > ****************************************
> >
>
>
>
> ------------------------------
>
> _______________________________________________
> General mailing list
> General@developer.marklogic.com
> Manage your subscription at:
> http://developer.marklogic.com/mailman/listinfo/general
>
>
> End of General Digest, Vol 147, Issue 53
> ****************************************
>
_______________________________________________
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
