You’re trying to update all of the documents in the “input” collection in a single transaction, the default scope of a JavaScript module. For small numbers of documents (hundreds or thousands) that will work. For large or unknown numbers of documents that will generally overwhelm some resource. In your case, you’ve blown out your expanded tree cache. Your best bet is to break the transformation into multiple transactions and spread those out over multiple hosts in a cluster. The MarkLogic Data Movement SDK is designed for exactly such workloads. Take a look at the docs <https://docs.marklogic.com/guide/java/data-movement#id_51555> for more information on how to orchestrate a bulk transformation from Java.
Justin -- Justin Makeig Senior Director, Product Management MarkLogic jmak...@marklogic.com<mailto:jmak...@marklogic.com> On Jan 24, 2018, at 10:19 AM, C. Yaswanth <rocking...@gmail.com<mailto:rocking...@gmail.com>> wrote: Hi All, Actually i have a set of json files(i.e.Total : 1M with Size 500MB). Each json file has 18 Keys. I tried to implement Envelope pattern using below Javascript 'use strict'; declareUpdate() var docs = fn.collection("input"); for(var doc of docs) { var transformed = {}; transformed.Metadata = { "Last Used" : ""}; transformed.Updated = { "University" : "UCLA"} transformed.Source = doc; //Sending original data under Source section xdmp.nodeReplace(doc,transformed) } I tried invoking this `JS.sjs` using JAVA API of marklogic 9. But i encountered below error : Exception in thread "main" com.marklogic.client.FailedRequestException: Local message: failed to apply resource at invoke: Internal Server Error. Server Message: XDMP-EXPNTREECACHEFULL: for(var doc of docs) { -- Expanded tree cache full on host localhost uri file.json-0-968991 at com.marklogic.client.impl.OkHttpServices.checkStatus(OkHttpServices.java:4317) at com.marklogic.client.impl.OkHttpServices.postIteratedResourceImpl(OkHttpServices.java:3831) at com.marklogic.client.impl.OkHttpServices.postEvalInvoke(OkHttpServices.java:3768) at com.marklogic.client.impl.ServerEvaluationCallImpl.eval(ServerEvaluationCallImpl.java:164) at com.marklogic.client.impl.ServerEvaluationCallImpl.eval(ServerEvaluationCallImpl.java:153) at com.marklogic.client.impl.ServerEvaluationCallImpl.evalAs(ServerEvaluationCallImpl.java:144) at bulkimport.Tsm.main(Tsm.java:19) I went through documentation (i.e.https://help.marklogic.com/knowledgebase/article/View/9/16/resolving-xdmp-expntreecachefull-errors) where they had mentioned ways to resolve this error. Following that i had increased `expanded tree cache size*` to `2048` but still i am facing same error. How can i optimize by above code (i.e.`JS.sjs`) to avoid this error ? Any help is appreciated. _______________________________________________ General mailing list General@developer.marklogic.com<mailto:General@developer.marklogic.com> Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
