[MarkLogic Dev General] Listing Directories?
I'm curious as to how certain directories get created in my database. I have:
23,135,073 documents
9,916 directories
I don't explicitly create directories in any of my import pipelines.
If I could list them (or some sampling of them), I might have a better
idea of where they are coming from.
In the documentation, there is an example:
for $x in xdmp:document-properties()/prop:properties/prop:directory
return directory-uri{xdmp:node-uri($x)}/directory-uri
but that is a terrible idea if you have millions of documents.
Is there a way to get the directories directly somehow?
--
--Alex Milowski
The excellence of grammar as a guide is proportional to the paucity of the
inflexions, i.e. to the degree of analysis effected by the language
considered.
Bertrand Russell in a footnote of Principles of Mathematics
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Hi Alex,
If you have a URI lexicon, you can look for URIs that end in /,
something like:
cts:uri-match(*/)
Also, if you just want to see how many directories there are you can do
something like this:
xdmp:estimate(xdmp:document-properties()/prop:properties/prop:directory)
-Danny
-Original Message-
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski
Sent: Thursday, July 05, 2012 11:27 AM
To: General Mark Logic Developer Discussion
Subject: [MarkLogic Dev General] Listing Directories?
I'm curious as to how certain directories get created in my database. I have:
23,135,073 documents
9,916 directories
I don't explicitly create directories in any of my import pipelines.
If I could list them (or some sampling of them), I might have a better
idea of where they are coming from.
In the documentation, there is an example:
for $x in xdmp:document-properties()/prop:properties/prop:directory
return directory-uri{xdmp:node-uri($x)}/directory-uri
but that is a terrible idea if you have millions of documents.
Is there a way to get the directories directly somehow?
--
--Alex Milowski
The excellence of grammar as a guide is proportional to the paucity of the
inflexions, i.e. to the degree of analysis effected by the language
considered.
Bertrand Russell in a footnote of Principles of Mathematics
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
You could try something like:
cts:uris((),properties, cts:properties-query(
cts:element-query(xs:QName(prop:directory),cts:and-query(()))
))
-Ryan Dew
On Thu, Jul 5, 2012 at 12:27 PM, Alex Milowski a...@milowski.com wrote:
I'm curious as to how certain directories get created in my database. I
have:
23,135,073 documents
9,916 directories
I don't explicitly create directories in any of my import pipelines.
If I could list them (or some sampling of them), I might have a better
idea of where they are coming from.
In the documentation, there is an example:
for $x in xdmp:document-properties()/prop:properties/prop:directory
return directory-uri{xdmp:node-uri($x)}/directory-uri
but that is a terrible idea if you have millions of documents.
Is there a way to get the directories directly somehow?
--
--Alex Milowski
The excellence of grammar as a guide is proportional to the paucity of the
inflexions, i.e. to the degree of analysis effected by the language
considered.
Bertrand Russell in a footnote of Principles of Mathematics
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Hmm... no URI lexicon so neither will work. I've been reluctant to turn that on because of the large number of documents. I never access the content by URI. On Thu, Jul 5, 2012 at 11:32 AM, Ryan Dew ryan.j@gmail.com wrote: You could try something like: cts:uris((),properties, cts:properties-query( cts:element-query(xs:QName(prop:directory),cts:and-query(())) )) -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
You don't need the URI lexicon for the xdmp:estimate one. -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:37 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? Hmm... no URI lexicon so neither will work. I've been reluctant to turn that on because of the large number of documents. I never access the content by URI. On Thu, Jul 5, 2012 at 11:32 AM, Ryan Dew ryan.j@gmail.com wrote: You could try something like: cts:uris((),properties, cts:properties-query( cts:element-query(xs:QName(prop:directory),cts:and-query(())) )) -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky danny.sokol...@marklogic.com wrote: You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Do you have directory creation set to automatic in your database settings? -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:48 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky danny.sokol...@marklogic.com wrote: You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
So why is the example from the documentation so bad? The expression is fully searchable and will only return directory properties, so having millions of documents shouldn't matter. The only count that should really matter are the number of directories. Then if you really wanted to you could paginate over the results using fn:subsequence. xdmp:plan(xdmp:document-properties()/prop:properties/prop:directory) and this might be slightly more efficient: xdmp:plan(xdmp:document-properties()/prop:properties[prop:directory]) On Thu, Jul 5, 2012 at 12:49 PM, Will Thompson wthomp...@jonesmcclure.comwrote: Do you have directory creation set to automatic in your database settings? -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto: general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:48 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky danny.sokol...@marklogic.com wrote: You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
It times out on 23 million documents for my current server configuration. I've reduced my database size to a more reasonable size but I still need a few more GB of real memory. As such, I suspect that query induces paging and slows everything down. On Thu, Jul 5, 2012 at 12:09 PM, Ryan Dew ryan.j@gmail.com wrote: So why is the example from the documentation so bad? The expression is fully searchable and will only return directory properties, so having millions of documents shouldn't matter. The only count that should really matter are the number of directories. Then if you really wanted to you could paginate over the results using fn:subsequence. xdmp:plan(xdmp:document-properties()/prop:properties/prop:directory) and this might be slightly more efficient: xdmp:plan(xdmp:document-properties()/prop:properties[prop:directory]) On Thu, Jul 5, 2012 at 12:49 PM, Will Thompson wthomp...@jonesmcclure.com wrote: Do you have directory creation set to automatic in your database settings? -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:48 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky danny.sokol...@marklogic.com wrote: You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Ah! Yes I do. That number of directories looks suspiciously close to the number of stations I have stored in my database. That would probably account for it. I'll never use WebDAV on this database give the large number of small documents, so I can turn that off. Is there any other reason why I would want that on? On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson wthomp...@jonesmcclure.com wrote: Do you have directory creation set to automatic in your database settings? -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:48 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky danny.sokol...@marklogic.com wrote: You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
To my knowledge, there is no advantage other than for WebDav. And you can still search URIs as if they are directories even if there are no directory fragments. -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 12:14 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? Ah! Yes I do. That number of directories looks suspiciously close to the number of stations I have stored in my database. That would probably account for it. I'll never use WebDAV on this database give the large number of small documents, so I can turn that off. Is there any other reason why I would want that on? On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson wthomp...@jonesmcclure.com wrote: Do you have directory creation set to automatic in your database settings? -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:48 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky danny.sokol...@marklogic.com wrote: You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
On Thu, Jul 5, 2012 at 12:09 PM, Ryan Dew ryan.j@gmail.com wrote: So why is the example from the documentation so bad? The expression is fully searchable and will only return directory properties, so having millions of documents shouldn't matter. The only count that should really matter are the number of directories. Then if you really wanted to you could paginate over the results using fn:subsequence. xdmp:plan(xdmp:document-properties()/prop:properties/prop:directory) and this might be slightly more efficient: xdmp:plan(xdmp:document-properties()/prop:properties[prop:directory]) Never mind. They're both working now. I don't know why it timed out before. Probably a merge operation was in progress. I'm trying to calculate the actual memory needed for the data set by loading about 3 weeks of data. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
It's useful for xdmp:directory-delete(), which deletes all the documents in the directory in one fell swoop (assuming it doesn't time out first). I also find it useful for doing ad hoc queries with xdmp:directory(), but you probably have other fields you can query on that may be equivalent. If you are removing unneeded stuff, you might also check whether you have maintain time last modified turned on. It can be useful, but if you don't need it I think you can save the need to create a properties document for every document if you turn that off and also turn off directory creation. You may have noticed that you have twice as many fragments as documents - that's why, I think. There might also be some savings from disabling inherit collections and inherit permissions if you never use those things. I'm less sure about that though. -Mike On 07/05/2012 03:14 PM, Alex Milowski wrote: Ah! Yes I do. That number of directories looks suspiciously close to the number of stations I have stored in my database. That would probably account for it. I'll never use WebDAV on this database give the large number of small documents, so I can turn that off. Is there any other reason why I would want that on? On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson wthomp...@jonesmcclure.com wrote: Do you have directory creation set to automatic in your database settings? -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:48 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolskydanny.sokol...@marklogic.com wrote: You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
I agree about maintain-last-modified, but directory fragments are not necessary for xdmp:directory-delete or xdmp:directory. http://blakeley.com/blogofile/2012/03/19/directory-assistance/ tries to explain that the database index includes special term entries for directories, whether the directory fragment exists or not. Once directory-creation=manual, you might want to delete any existing directory fragments. xdmp:document-properties()/prop:properties[prop:directory]/xdmp:node-delete(.) If you have a large number of directories, it may be easier to delete a few at a time: (xdmp:document-properties()/prop:properties[prop:directory])[1 to 1000]/xdmp:node-delete(.) The URI lexicon would make that easier too, but only if you already have it enabled. -- Mike On 5 Jul 2012, at 12:50 , Mike Sokolov wrote: It's useful for xdmp:directory-delete(), which deletes all the documents in the directory in one fell swoop (assuming it doesn't time out first). I also find it useful for doing ad hoc queries with xdmp:directory(), but you probably have other fields you can query on that may be equivalent. If you are removing unneeded stuff, you might also check whether you have maintain time last modified turned on. It can be useful, but if you don't need it I think you can save the need to create a properties document for every document if you turn that off and also turn off directory creation. You may have noticed that you have twice as many fragments as documents - that's why, I think. There might also be some savings from disabling inherit collections and inherit permissions if you never use those things. I'm less sure about that though. -Mike On 07/05/2012 03:14 PM, Alex Milowski wrote: Ah! Yes I do. That number of directories looks suspiciously close to the number of stations I have stored in my database. That would probably account for it. I'll never use WebDAV on this database give the large number of small documents, so I can turn that off. Is there any other reason why I would want that on? On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson wthomp...@jonesmcclure.com wrote: Do you have directory creation set to automatic in your database settings? -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:48 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolskydanny.sokol...@marklogic.com wrote: You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
You do not need automatic directory creation for xdmp:directory-delete or for xdmp:directory. If you are not using WebDAV, there is no real need for automatic directory creation. Similarly, if you are not using the last-modified property, there is often no need for maintain last modified. -Danny -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Mike Sokolov Sent: Thursday, July 05, 2012 12:51 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? It's useful for xdmp:directory-delete(), which deletes all the documents in the directory in one fell swoop (assuming it doesn't time out first). I also find it useful for doing ad hoc queries with xdmp:directory(), but you probably have other fields you can query on that may be equivalent. If you are removing unneeded stuff, you might also check whether you have maintain time last modified turned on. It can be useful, but if you don't need it I think you can save the need to create a properties document for every document if you turn that off and also turn off directory creation. You may have noticed that you have twice as many fragments as documents - that's why, I think. There might also be some savings from disabling inherit collections and inherit permissions if you never use those things. I'm less sure about that though. -Mike On 07/05/2012 03:14 PM, Alex Milowski wrote: Ah! Yes I do. That number of directories looks suspiciously close to the number of stations I have stored in my database. That would probably account for it. I'll never use WebDAV on this database give the large number of small documents, so I can turn that off. Is there any other reason why I would want that on? On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson wthomp...@jonesmcclure.com wrote: Do you have directory creation set to automatic in your database settings? -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:48 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolskydanny.sokol...@marklogic.com wrote: You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
On Thu, Jul 5, 2012 at 12:20 PM, Will Thompson wthomp...@jonesmcclure.com wrote: To my knowledge, there is no advantage other than for WebDav. And you can still search URIs as if they are directories even if there are no directory fragments. In that area of the database admin configuration page, there are a number of settings, if I set directory creation to manual, what else should I change? It looks to me like without directories being created automatically, I can safely ignore most of these settings. I currently have: maintain last modified : true maintain directory last modified: false inherit permissions: true inherit collections: true inherit quality: true -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Oh! Thanks for the clarification (and you too Danny - pity you can't reply to two messages at once). I have some catch-up reading to do I guess ... On 07/05/2012 03:54 PM, Michael Blakeley wrote: I agree about maintain-last-modified, but directory fragments are not necessary for xdmp:directory-delete or xdmp:directory. http://blakeley.com/blogofile/2012/03/19/directory-assistance/ tries to explain that the database index includes special term entries for directories, whether the directory fragment exists or not. Once directory-creation=manual, you might want to delete any existing directory fragments. xdmp:document-properties()/prop:properties[prop:directory]/xdmp:node-delete(.) If you have a large number of directories, it may be easier to delete a few at a time: (xdmp:document-properties()/prop:properties[prop:directory])[1 to 1000]/xdmp:node-delete(.) The URI lexicon would make that easier too, but only if you already have it enabled. -- Mike On 5 Jul 2012, at 12:50 , Mike Sokolov wrote: It's useful for xdmp:directory-delete(), which deletes all the documents in the directory in one fell swoop (assuming it doesn't time out first). I also find it useful for doing ad hoc queries with xdmp:directory(), but you probably have other fields you can query on that may be equivalent. If you are removing unneeded stuff, you might also check whether you have maintain time last modified turned on. It can be useful, but if you don't need it I think you can save the need to create a properties document for every document if you turn that off and also turn off directory creation. You may have noticed that you have twice as many fragments as documents - that's why, I think. There might also be some savings from disabling inherit collections and inherit permissions if you never use those things. I'm less sure about that though. -Mike On 07/05/2012 03:14 PM, Alex Milowski wrote: Ah! Yes I do. That number of directories looks suspiciously close to the number of stations I have stored in my database. That would probably account for it. I'll never use WebDAV on this database give the large number of small documents, so I can turn that off. Is there any other reason why I would want that on? On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson wthomp...@jonesmcclure.com wrote: Do you have directory creation set to automatic in your database settings? -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:48 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolskydanny.sokol...@marklogic.com wrote: You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Once directories are created, I don't suppose there is anyway to get rid of them without getting rid of the documents? xdmp:directory-delete() removes the contained document (as you might expect). I structured everything based on collection membership and not directories. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Alex, see Mike's post from below. -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 2:24 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? Once directories are created, I don't suppose there is anyway to get rid of them without getting rid of the documents? xdmp:directory-delete() removes the contained document (as you might expect). I structured everything based on collection membership and not directories. -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
On Thu, Jul 5, 2012 at 2:31 PM, Will Thompson wthomp...@jonesmcclure.com wrote: Alex, see Mike's post from below. Yes, OK. I get it now. That's a bit obscure but it works quite well (and fast). I actually tested this all and re-created a single directory just to see how the documents get reassigned by URI subsumption. I now makes a lot more sense. Thanks! -- --Alex Milowski The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered. Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
