Hi Folks,
The MarkLogic documentation clearly states that: The main purpose of a WebDAV server is to make it easy for people to store, retrieve, and modify documents in a database. The documents can be any type, whether they are text documents such as .txt files or source code, binary documents such as image files or Microsoft Word files, or XML documents. Because the documents are stored in a database, you can create applications that use the content in those documents for whatever purpose you need. You can also use the database backup and restore features to easily back up the content in the database. WebDAV is pretty useless when it comes to browsing directory URIs than contain too many documents and/or subdirectories, but I have been operating on the assumption that by organizing a large set of XML documents in a hierarchical directory URI structure that I can limit the number of documents and subdirectories that are accessible via the Data Source Explorer in oXygen's WebDAV client (and other WebDAV clients as well). It has been extremely valuable for me to use the oXygen WebDAV Data Source Explorer to quickly drill down, locate documents, compare the input and output of transforms, to debug updated transforms, and when necessary to manually correct and save an errant XML document in the input stream of a CPF pipeline. This functionality is not simply available in CQ. However, I was recently informed that even if directory structures are organized hierarchically, that WebDAV clients still cause the server to incur significant performance hits when opening a directory in the oXygen or other WebDAV treeview to explore its contents. The risk may be more pronounced on a production system than on a development system, but full content sets for evaluation may only be available in a production environment. I haven't seen any documentation that discusses these details and I'm not familiar with the WebDAV API's for browsing and reading directory URIs in MarkLogic, so I would like to go a little deeper and try to determine: 1. How much of a performance hit is incurred and at what point, i.e., when there are 1,000 subdirectories and/or document URIs within a given directory URI? Is there a given number at which the performance hit becomes negligible so that using a hierarchical directory URI structure is feasible? Is there a way to measure that performance for any given WebDAV client? 2. Can the WebDAV client be tuned to explicitly prevent any significant hits to the server, i.e., by limiting threads, timeouts, etc? 3. The 4.1 documentation refers to tested WebDAV clients. I'm surprised that list doesn't include oXygen or some other freeware clients. I'm assuming that by saying that the listed WebDAV clients were tested that they also passed some form of acceptance testing. I'd like to suggest that the list be updated to reflect oXygen, EnGinSite DataFreeway, and BitKinex. 4. Are there protocols in WebDAV that allow for limited directory viewing - that is to only request the first N subdirectories and/or documents within a given directory URI so as to explicitly limit the load on the server when trying to get a directory listing? My options for replacing the use of oXygen and to avoid performance penalties associated with WebDAV are to: 1. Build my own WebDAV client (if I can limit the directory listings), or 2. Build an xquery web app that uses optimized queries such as cts:uris to drill down into directory URIs with an expandable TreeView to locate documents (and as a means of using the oXygen WebDAV-based editor to create the URL of the document I want to open and to paste it into the oXygen OpenURL feature). 3. Find existing code that can hopefully be ported to perform either of the above. Thank you all for any help with this! Tim Meagher
_______________________________________________ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
