Hi Gary, A few thoughts here. You are using 7.0-4 on this?
What are you comparing it to on the Oracle side? In MarkLogic, the content will be all indexed and searchable. Is that true on the orcl side too? What indexes to you have enabled? Maybe you do not need them all (or maybe you should put the equivalent indexing on the orcl side)? Have you tried tweaking the batch size? I would try a smaller number, say 50 or 100. Have you analyzed where you are spending the time? In the c# code? In the code loading the doc on MarkLogic? Do you have multiple threads loading from your .net program? If you are not maxing out your cpu on the MarkLogic side, you probably have room for more parallelization. -Danny From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Gary Russo Sent: Tuesday, October 14, 2014 9:21 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] How to optimize the REST API Bulk Ingestion Performance? MarkLogic Bulk ingestion processing is slower than an equivalent Oracle ingestion process. The MarkLogic ingestion takes 30 minutes. An Oracle equivalent only takes 7 minutes. I'm using the REST API to bulk ingest multiple documents as described here. => http://docs.marklogic.com/guide/rest-dev/bulk#id_54649 Notes: * C# code is used to call the MarkLogic Bulk Ingest REST API. * Document batch size used is 500. * Average doc size is 1 KB. * JSON Conversion and Validation logic occurs in the C# code. Any thoughts on how to optimize the MarkLogic bulk ingest to make it as fast as Oracle's 7 minute load time? Thanks, Gary R Gary Russo Enterprise NoSQL Developer http://garyrusso.wordpress.com
_______________________________________________ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
