If I understand you correctly, you have 5067188 docs - what is an approx size of each ? Parsing occurs only during document ingest so parse time will not affect search time. If you added fragmentation on that basis it is likely to hurt rather than help ( in general you should not use fragmentation unless you have large documents with very distinct sub structure - and then many of the APIs will behave in ways you may not expect).
What is the size of your result set ? Generally you want pagination to keep the result set small for each page ... if your sending millions of snippets in the result the client app has to buffer all of that first. If possible, try to separate the performance analysis into the queries vs the application. If you can run the query directly in Console you can use the Profiler to help determine where the bottlenecks are. A good place to start https://docs.marklogic.com/guide/performance/profile https://help.marklogic.com/knowledgebase/article/View/48/17/how-do-i-read-and-interpret-qconsole-profile-output Regardless of which API you use (java, xquery , javascript etc) the same server search code is run, so its useful to be able to profile and debug at the server level either with QCOnsole or the Profile API - the results from that can be applied to any of the client API's Very often, (until you have refined your data model and queries) long searches with many docs are due to queries being able to resolve via the indexes. An efficient search should return results quickie even with very large number of number of documents - as long as the queries are able to make use of the indexes. This can be improved by both adjusting the queries and/or the indexes used. Its much easier to do that when you have the profiling data to see where the time is spent or you can take a very long time randomly guessing. I suggest first, removing the fragmentation (and letting the DB reindex) then starting with the simplest query that is not performing well use one or more of the profiling techniques. Likely you will identify a bottleneck very quickly - that will help you decide how to adjust the query or the indexing or both. ----------------------------------------------------------------------------- David Lee Lead Engineer MarkLogic Corporation d...@marklogic.com Phone: +1 812-482-5224 Cell: +1 812-630-7622 www.marklogic.com<http://www.marklogic.com/> From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Shashidhar Rao Sent: Saturday, May 02, 2015 7:00 PM To: general@developer.marklogic.com Subject: Re: [MarkLogic Dev General] Fragment root configuration Thanks for your reply Christopher and David. Content is xml . The search app is taking rough 5.6 secs to show the contents snippets , facets etc with about 5 million records. Query used is JAVA API using QueryManager search function. Read that fragment root keeps each document as one so I thought it could be faster during the parsing stage, but haven't found anywhere to validate this one. Pagination is a bit slow. All others are fine like details, facets links and others. Only the QueryManager search takes long time. Thanks On Sat, May 2, 2015 at 6:08 AM, Shashidhar Rao <raoshashidhar...@gmail.com<mailto:raoshashidhar...@gmail.com>> wrote: Hi, Will configuring Fragment roots speed up my query. I am building a search application using Marklogic Java api and I have around 6 different root elements. My search app takes around 5 seconds , to search and to show the snippets and around 4 filters with roughly 5067188 docs. I read that Fragment roots make it one continuous compressed binary document will it speed up my search. Kindly suggest. Thanks
_______________________________________________ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
