Re: [MarkLogic Dev General] Listing Directories?
On Thu, Jul 5, 2012 at 2:31 PM, Will Thompson wrote: > Alex, see Mike's post from below. > Yes, OK. I get it now. That's a bit obscure but it works quite well (and fast). I actually tested this all and re-created a single directory just to see how the documents get reassigned by URI subsumption. I now makes a lot more sense. Thanks! -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Alex, see Mike's post from below. -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 2:24 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? Once directories are created, I don't suppose there is anyway to get rid of them without getting rid of the documents? xdmp:directory-delete() removes the contained document (as you might expect). I structured everything based on collection membership and not directories. -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Once directories are created, I don't suppose there is anyway to get rid of them without getting rid of the documents? xdmp:directory-delete() removes the contained document (as you might expect). I structured everything based on collection membership and not directories. -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Oh! Thanks for the clarification (and you too Danny - pity you can't reply to two messages at once). I have some catch-up reading to do I guess ... On 07/05/2012 03:54 PM, Michael Blakeley wrote: > I agree about maintain-last-modified, but directory fragments are not > necessary for xdmp:directory-delete or xdmp:directory. > http://blakeley.com/blogofile/2012/03/19/directory-assistance/ tries to > explain that the database index includes special term entries for > directories, whether the directory fragment exists or not. > > Once directory-creation=manual, you might want to delete any existing > directory fragments. > > > xdmp:document-properties()/prop:properties[prop:directory]/xdmp:node-delete(.) > > If you have a large number of directories, it may be easier to delete a few > at a time: > > (xdmp:document-properties()/prop:properties[prop:directory])[1 to > 1000]/xdmp:node-delete(.) > > The URI lexicon would make that easier too, but only if you already have it > enabled. > > -- Mike > > On 5 Jul 2012, at 12:50 , Mike Sokolov wrote: > > >> It's useful for xdmp:directory-delete(), which deletes all the documents >> in the directory in one fell swoop (assuming it doesn't time out first). >> >> I also find it useful for doing ad hoc queries with xdmp:directory(), >> but you probably have other fields you can query on that may be equivalent. >> >> If you are removing unneeded stuff, you might also check whether you >> have "maintain time last modified" turned on. It can be useful, but if >> you don't need it I think you can save the need to create a properties >> document for every document if you turn that off and also turn off >> directory creation. You may have noticed that you have twice as many >> fragments as documents - that's why, I think. >> >> There might also be some savings from disabling "inherit collections" >> and "inherit permissions" if you never use those things. I'm less sure >> about that though. >> >> -Mike >> >> On 07/05/2012 03:14 PM, Alex Milowski wrote: >> >>> Ah! Yes I do. That number of directories looks suspiciously close to >>> the number of stations I have stored in my database. That would >>> probably account for it. >>> >>> I'll never use WebDAV on this database give the large number of small >>> documents, so I can turn that off. Is there any other reason why I >>> would want that on? >>> >>> On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson >>>wrote: >>> >>> Do you have directory creation set to automatic in your database settings? -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:48 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky wrote: > You don't need the URI lexicon for the xdmp:estimate one. > > OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general >>> >>> >>> >> ___ >> General mailing list >> General@developer.marklogic.com >> http://developer.marklogic.com/mailman/listinfo/general >> >> > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general > ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
On Thu, Jul 5, 2012 at 12:20 PM, Will Thompson wrote: > To my knowledge, there is no advantage other than for WebDav. And you can > still search URIs as if they are directories even if there are no directory > fragments. In that area of the database admin configuration page, there are a number of settings, if I set directory creation to manual, what else should I change? It looks to me like without directories being created automatically, I can safely ignore most of these settings. I currently have: maintain last modified : true maintain directory last modified: false inherit permissions: true inherit collections: true inherit quality: true -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
You do not need automatic directory creation for xdmp:directory-delete or for xdmp:directory. If you are not using WebDAV, there is no real need for automatic directory creation. Similarly, if you are not using the last-modified property, there is often no need for maintain last modified. -Danny -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Mike Sokolov Sent: Thursday, July 05, 2012 12:51 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? It's useful for xdmp:directory-delete(), which deletes all the documents in the directory in one fell swoop (assuming it doesn't time out first). I also find it useful for doing ad hoc queries with xdmp:directory(), but you probably have other fields you can query on that may be equivalent. If you are removing unneeded stuff, you might also check whether you have "maintain time last modified" turned on. It can be useful, but if you don't need it I think you can save the need to create a properties document for every document if you turn that off and also turn off directory creation. You may have noticed that you have twice as many fragments as documents - that's why, I think. There might also be some savings from disabling "inherit collections" and "inherit permissions" if you never use those things. I'm less sure about that though. -Mike On 07/05/2012 03:14 PM, Alex Milowski wrote: > Ah! Yes I do. That number of directories looks suspiciously close to > the number of stations I have stored in my database. That would > probably account for it. > > I'll never use WebDAV on this database give the large number of small > documents, so I can turn that off. Is there any other reason why I > would want that on? > > On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson > wrote: > >> Do you have directory creation set to automatic in your database settings? >> >> -Will >> >> -Original Message- >> From: general-boun...@developer.marklogic.com >> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski >> Sent: Thursday, July 05, 2012 11:48 AM >> To: MarkLogic Developer Discussion >> Subject: Re: [MarkLogic Dev General] Listing Directories? >> >> On Thu, Jul 5, 2012 at 11:37 AM, Danny >> Sokolsky wrote: >> >>> You don't need the URI lexicon for the xdmp:estimate one. >>> >> OK. Good to know. >> >> What I really want to know is the name and purpose of these directories >> since I don't actually create them explicitly. >> >> -- >> --Alex Milowski >> "The excellence of grammar as a guide is proportional to the paucity of the >> inflexions, i.e. to the degree of analysis effected by the language >> considered." >> >> Bertrand Russell in a footnote of Principles of Mathematics >> ___ >> General mailing list >> General@developer.marklogic.com >> http://developer.marklogic.com/mailman/listinfo/general >> ___ >> General mailing list >> General@developer.marklogic.com >> http://developer.marklogic.com/mailman/listinfo/general >> > > > ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
I agree about maintain-last-modified, but directory fragments are not necessary for xdmp:directory-delete or xdmp:directory. http://blakeley.com/blogofile/2012/03/19/directory-assistance/ tries to explain that the database index includes special term entries for directories, whether the directory fragment exists or not. Once directory-creation=manual, you might want to delete any existing directory fragments. xdmp:document-properties()/prop:properties[prop:directory]/xdmp:node-delete(.) If you have a large number of directories, it may be easier to delete a few at a time: (xdmp:document-properties()/prop:properties[prop:directory])[1 to 1000]/xdmp:node-delete(.) The URI lexicon would make that easier too, but only if you already have it enabled. -- Mike On 5 Jul 2012, at 12:50 , Mike Sokolov wrote: > It's useful for xdmp:directory-delete(), which deletes all the documents > in the directory in one fell swoop (assuming it doesn't time out first). > > I also find it useful for doing ad hoc queries with xdmp:directory(), > but you probably have other fields you can query on that may be equivalent. > > If you are removing unneeded stuff, you might also check whether you > have "maintain time last modified" turned on. It can be useful, but if > you don't need it I think you can save the need to create a properties > document for every document if you turn that off and also turn off > directory creation. You may have noticed that you have twice as many > fragments as documents - that's why, I think. > > There might also be some savings from disabling "inherit collections" > and "inherit permissions" if you never use those things. I'm less sure > about that though. > > -Mike > > On 07/05/2012 03:14 PM, Alex Milowski wrote: >> Ah! Yes I do. That number of directories looks suspiciously close to >> the number of stations I have stored in my database. That would >> probably account for it. >> >> I'll never use WebDAV on this database give the large number of small >> documents, so I can turn that off. Is there any other reason why I >> would want that on? >> >> On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson >> wrote: >> >>> Do you have directory creation set to automatic in your database settings? >>> >>> -Will >>> >>> -Original Message- >>> From: general-boun...@developer.marklogic.com >>> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski >>> Sent: Thursday, July 05, 2012 11:48 AM >>> To: MarkLogic Developer Discussion >>> Subject: Re: [MarkLogic Dev General] Listing Directories? >>> >>> On Thu, Jul 5, 2012 at 11:37 AM, Danny >>> Sokolsky wrote: >>> You don't need the URI lexicon for the xdmp:estimate one. >>> OK. Good to know. >>> >>> What I really want to know is the name and purpose of these directories >>> since I don't actually create them explicitly. >>> >>> -- >>> --Alex Milowski >>> "The excellence of grammar as a guide is proportional to the paucity of the >>> inflexions, i.e. to the degree of analysis effected by the language >>> considered." >>> >>> Bertrand Russell in a footnote of Principles of Mathematics >>> ___ >>> General mailing list >>> General@developer.marklogic.com >>> http://developer.marklogic.com/mailman/listinfo/general >>> ___ >>> General mailing list >>> General@developer.marklogic.com >>> http://developer.marklogic.com/mailman/listinfo/general >>> >> >> >> > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general > ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
It's useful for xdmp:directory-delete(), which deletes all the documents in the directory in one fell swoop (assuming it doesn't time out first). I also find it useful for doing ad hoc queries with xdmp:directory(), but you probably have other fields you can query on that may be equivalent. If you are removing unneeded stuff, you might also check whether you have "maintain time last modified" turned on. It can be useful, but if you don't need it I think you can save the need to create a properties document for every document if you turn that off and also turn off directory creation. You may have noticed that you have twice as many fragments as documents - that's why, I think. There might also be some savings from disabling "inherit collections" and "inherit permissions" if you never use those things. I'm less sure about that though. -Mike On 07/05/2012 03:14 PM, Alex Milowski wrote: > Ah! Yes I do. That number of directories looks suspiciously close to > the number of stations I have stored in my database. That would > probably account for it. > > I'll never use WebDAV on this database give the large number of small > documents, so I can turn that off. Is there any other reason why I > would want that on? > > On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson > wrote: > >> Do you have directory creation set to automatic in your database settings? >> >> -Will >> >> -Original Message- >> From: general-boun...@developer.marklogic.com >> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski >> Sent: Thursday, July 05, 2012 11:48 AM >> To: MarkLogic Developer Discussion >> Subject: Re: [MarkLogic Dev General] Listing Directories? >> >> On Thu, Jul 5, 2012 at 11:37 AM, Danny >> Sokolsky wrote: >> >>> You don't need the URI lexicon for the xdmp:estimate one. >>> >> OK. Good to know. >> >> What I really want to know is the name and purpose of these directories >> since I don't actually create them explicitly. >> >> -- >> --Alex Milowski >> "The excellence of grammar as a guide is proportional to the paucity of the >> inflexions, i.e. to the degree of analysis effected by the language >> considered." >> >> Bertrand Russell in a footnote of Principles of Mathematics >> ___ >> General mailing list >> General@developer.marklogic.com >> http://developer.marklogic.com/mailman/listinfo/general >> ___ >> General mailing list >> General@developer.marklogic.com >> http://developer.marklogic.com/mailman/listinfo/general >> > > > ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
On Thu, Jul 5, 2012 at 12:09 PM, Ryan Dew wrote: > So why is the example from the documentation so bad? The expression is fully > searchable and will only return directory properties, so having millions of > documents shouldn't matter. The only count that should really matter are the > number of directories. Then if you really wanted to you could paginate over > the results using fn:subsequence. > > xdmp:plan(xdmp:document-properties()/prop:properties/prop:directory) > > and this might be slightly more efficient: > > xdmp:plan(xdmp:document-properties()/prop:properties[prop:directory]) Never mind. They're both working now. I don't know why it timed out before. Probably a merge operation was in progress. I'm trying to calculate the actual memory needed for the data set by loading about 3 weeks of data. -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
To my knowledge, there is no advantage other than for WebDav. And you can still search URIs as if they are directories even if there are no directory fragments. -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 12:14 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? Ah! Yes I do. That number of directories looks suspiciously close to the number of stations I have stored in my database. That would probably account for it. I'll never use WebDAV on this database give the large number of small documents, so I can turn that off. Is there any other reason why I would want that on? On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson wrote: > Do you have directory creation set to automatic in your database settings? > > -Will > > -Original Message- > From: general-boun...@developer.marklogic.com > [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex > Milowski > Sent: Thursday, July 05, 2012 11:48 AM > To: MarkLogic Developer Discussion > Subject: Re: [MarkLogic Dev General] Listing Directories? > > On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky > wrote: >> You don't need the URI lexicon for the xdmp:estimate one. > > OK. Good to know. > > What I really want to know is the name and purpose of these directories since > I don't actually create them explicitly. > > -- > --Alex Milowski > "The excellence of grammar as a guide is proportional to the paucity of the > inflexions, i.e. to the degree of analysis effected by the language > considered." > > Bertrand Russell in a footnote of Principles of Mathematics > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Ah! Yes I do. That number of directories looks suspiciously close to the number of stations I have stored in my database. That would probably account for it. I'll never use WebDAV on this database give the large number of small documents, so I can turn that off. Is there any other reason why I would want that on? On Thu, Jul 5, 2012 at 11:49 AM, Will Thompson wrote: > Do you have directory creation set to automatic in your database settings? > > -Will > > -Original Message- > From: general-boun...@developer.marklogic.com > [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski > Sent: Thursday, July 05, 2012 11:48 AM > To: MarkLogic Developer Discussion > Subject: Re: [MarkLogic Dev General] Listing Directories? > > On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky > wrote: >> You don't need the URI lexicon for the xdmp:estimate one. > > OK. Good to know. > > What I really want to know is the name and purpose of these directories since > I don't actually create them explicitly. > > -- > --Alex Milowski > "The excellence of grammar as a guide is proportional to the paucity of the > inflexions, i.e. to the degree of analysis effected by the language > considered." > > Bertrand Russell in a footnote of Principles of Mathematics > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
It times out on 23 million documents for my current server configuration. I've reduced my database size to a more reasonable size but I still need a few more GB of real memory. As such, I suspect that query induces paging and slows everything down. On Thu, Jul 5, 2012 at 12:09 PM, Ryan Dew wrote: > So why is the example from the documentation so bad? The expression is fully > searchable and will only return directory properties, so having millions of > documents shouldn't matter. The only count that should really matter are the > number of directories. Then if you really wanted to you could paginate over > the results using fn:subsequence. > > xdmp:plan(xdmp:document-properties()/prop:properties/prop:directory) > > and this might be slightly more efficient: > > xdmp:plan(xdmp:document-properties()/prop:properties[prop:directory]) > > > On Thu, Jul 5, 2012 at 12:49 PM, Will Thompson > wrote: >> >> Do you have directory creation set to automatic in your database settings? >> >> -Will >> >> -Original Message- >> From: general-boun...@developer.marklogic.com >> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski >> Sent: Thursday, July 05, 2012 11:48 AM >> To: MarkLogic Developer Discussion >> Subject: Re: [MarkLogic Dev General] Listing Directories? >> >> On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky >> wrote: >> > You don't need the URI lexicon for the xdmp:estimate one. >> >> OK. Good to know. >> >> What I really want to know is the name and purpose of these directories >> since I don't actually create them explicitly. >> >> -- >> --Alex Milowski >> "The excellence of grammar as a guide is proportional to the paucity of >> the inflexions, i.e. to the degree of analysis effected by the language >> considered." >> >> Bertrand Russell in a footnote of Principles of Mathematics >> ___ >> General mailing list >> General@developer.marklogic.com >> http://developer.marklogic.com/mailman/listinfo/general >> ___ >> General mailing list >> General@developer.marklogic.com >> http://developer.marklogic.com/mailman/listinfo/general > > > > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general > -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
So why is the example from the documentation so bad? The expression is fully searchable and will only return directory properties, so having millions of documents shouldn't matter. The only count that should really matter are the number of directories. Then if you really wanted to you could paginate over the results using fn:subsequence. xdmp:plan(xdmp:document-properties()/prop:properties/prop:directory) and this might be slightly more efficient: xdmp:plan(xdmp:document-properties()/prop:properties[prop:directory]) On Thu, Jul 5, 2012 at 12:49 PM, Will Thompson wrote: > Do you have directory creation set to automatic in your database settings? > > -Will > > -Original Message- > From: general-boun...@developer.marklogic.com [mailto: > general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski > Sent: Thursday, July 05, 2012 11:48 AM > To: MarkLogic Developer Discussion > Subject: Re: [MarkLogic Dev General] Listing Directories? > > On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky < > danny.sokol...@marklogic.com> wrote: > > You don't need the URI lexicon for the xdmp:estimate one. > > OK. Good to know. > > What I really want to know is the name and purpose of these directories > since I don't actually create them explicitly. > > -- > --Alex Milowski > "The excellence of grammar as a guide is proportional to the paucity of > the inflexions, i.e. to the degree of analysis effected by the language > considered." > > Bertrand Russell in a footnote of Principles of Mathematics > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general > ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Do you have directory creation set to automatic in your database settings? -Will -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Thursday, July 05, 2012 11:48 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Listing Directories? On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky wrote: > You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
On Thu, Jul 5, 2012 at 11:37 AM, Danny Sokolsky wrote: > You don't need the URI lexicon for the xdmp:estimate one. OK. Good to know. What I really want to know is the name and purpose of these directories since I don't actually create them explicitly. -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
You don't need the URI lexicon for the xdmp:estimate one.
-Original Message-
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski
Sent: Thursday, July 05, 2012 11:37 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Listing Directories?
Hmm... no URI lexicon so neither will work. I've been reluctant to
turn that on because of the large number of documents.
I never access the content by URI.
On Thu, Jul 5, 2012 at 11:32 AM, Ryan Dew wrote:
> You could try something like:
>
> cts:uris((),"properties", cts:properties-query(
>
> cts:element-query(xs:QName("prop:directory"),cts:and-query(()))
> ))
>
--
--Alex Milowski
"The excellence of grammar as a guide is proportional to the paucity of the
inflexions, i.e. to the degree of analysis effected by the language
considered."
Bertrand Russell in a footnote of Principles of Mathematics
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Hmm... no URI lexicon so neither will work. I've been reluctant to
turn that on because of the large number of documents.
I never access the content by URI.
On Thu, Jul 5, 2012 at 11:32 AM, Ryan Dew wrote:
> You could try something like:
>
> cts:uris((),"properties", cts:properties-query(
>
> cts:element-query(xs:QName("prop:directory"),cts:and-query(()))
> ))
>
--
--Alex Milowski
"The excellence of grammar as a guide is proportional to the paucity of the
inflexions, i.e. to the degree of analysis effected by the language
considered."
Bertrand Russell in a footnote of Principles of Mathematics
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
You could try something like:
cts:uris((),"properties", cts:properties-query(
cts:element-query(xs:QName("prop:directory"),cts:and-query(()))
))
-Ryan Dew
On Thu, Jul 5, 2012 at 12:27 PM, Alex Milowski wrote:
> I'm curious as to how certain directories get created in my database. I
> have:
>
>23,135,073 documents
>9,916 directories
>
> I don't explicitly create directories in any of my import pipelines.
>
> If I could list them (or some sampling of them), I might have a better
> idea of where they are coming from.
>
> In the documentation, there is an example:
>
>for $x in xdmp:document-properties()/prop:properties/prop:directory
> return {xdmp:node-uri($x)}
>
> but that is a terrible idea if you have millions of documents.
>
> Is there a way to get the directories directly somehow?
>
> --
> --Alex Milowski
> "The excellence of grammar as a guide is proportional to the paucity of the
> inflexions, i.e. to the degree of analysis effected by the language
> considered."
>
> Bertrand Russell in a footnote of Principles of Mathematics
> ___
> General mailing list
> General@developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general
>
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Listing Directories?
Hi Alex,
If you have a URI lexicon, you can look for URIs that end in "/",
something like:
cts:uri-match("*/")
Also, if you just want to see how many directories there are you can do
something like this:
xdmp:estimate(xdmp:document-properties()/prop:properties/prop:directory)
-Danny
-Original Message-
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski
Sent: Thursday, July 05, 2012 11:27 AM
To: General Mark Logic Developer Discussion
Subject: [MarkLogic Dev General] Listing Directories?
I'm curious as to how certain directories get created in my database. I have:
23,135,073 documents
9,916 directories
I don't explicitly create directories in any of my import pipelines.
If I could list them (or some sampling of them), I might have a better
idea of where they are coming from.
In the documentation, there is an example:
for $x in xdmp:document-properties()/prop:properties/prop:directory
return {xdmp:node-uri($x)}
but that is a terrible idea if you have millions of documents.
Is there a way to get the directories directly somehow?
--
--Alex Milowski
"The excellence of grammar as a guide is proportional to the paucity of the
inflexions, i.e. to the degree of analysis effected by the language
considered."
Bertrand Russell in a footnote of Principles of Mathematics
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
[MarkLogic Dev General] Listing Directories?
I'm curious as to how certain directories get created in my database. I have:
23,135,073 documents
9,916 directories
I don't explicitly create directories in any of my import pipelines.
If I could list them (or some sampling of them), I might have a better
idea of where they are coming from.
In the documentation, there is an example:
for $x in xdmp:document-properties()/prop:properties/prop:directory
return {xdmp:node-uri($x)}
but that is a terrible idea if you have millions of documents.
Is there a way to get the directories directly somehow?
--
--Alex Milowski
"The excellence of grammar as a guide is proportional to the paucity of the
inflexions, i.e. to the degree of analysis effected by the language
considered."
Bertrand Russell in a footnote of Principles of Mathematics
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
[MarkLogic Dev General] List server error
Hi All, I am getting an "Error: No address given" message when trying to log in to the list server to change preferences. (And I am giving an address.) Best, Tim Finney ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] search:search matching path coherence
Hi Stephane,
Several people have raised the same question. The behavior comes from
"stepping up" to a containing element in cases where an element contains an
exact match (works well for mixed content, but provides no benefit in this
particular case). I agree that the inconsistency is confusing. I'll file a bug
to take a look and see if the behavior can be improved for this case.
--Colleen
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] On Behalf Of Stephane Toussaint
[stephane.toussa...@4dconcept.fr]
Sent: Thursday, July 05, 2012 3:46 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] search:search matching path coherence
Hi,
I just find something weird with the matching path returned by the search API.
Say I have this xml in my repository
Toussaint
Stephane
If I request this document based on a firstname constraints the document is
well returned, but the matching path lake of precision :
import module namespace search="http://marklogic.com/appservices/search"; at
"/MarkLogic/appservices/search/search.xqy";
let $options := http://marklogic.com/appservices/search";>
return search:search("ste*", $options)
http://marklogic.com/appservices/search";>
Stephane
[...]
I expect the path to return
fn:doc("/content/article/test.xml")/ARTICLE/AUTHOR/FIRSTNAME but in
fact it stops on step upper.
But there is inconsistency here because if my firstname was a composed one (2
words), then the returned base is just not the same :
Toussaint
Jean Pierre
http://marklogic.com/appservices/search";>
Jean
Pierre
[...]
Is it a bug or can someone explain me why search result print different
information base on word count from element ?
By the way there is in fact no need of constraints to reproduce this exemple a
simple full text search provide the very same result.
I'd like to provide a filter step based on the path information, and this issue
did't allow me to do this.
Best Regards
Stephane
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
[MarkLogic Dev General] search:search matching path coherence
Hi,
I just find something weird with the matching path returned by the search API.
Say I have this xml in my repository
Toussaint
Stephane
If I request this document based on a firstname constraints the document is
well returned, but the matching path lake of precision :
import module namespace search="http://marklogic.com/appservices/search"; at
"/MarkLogic/appservices/search/search.xqy";
let $options := http://marklogic.com/appservices/search";>
return search:search("ste*", $options)
http://marklogic.com/appservices/search";>
Stephane
[...]
I expect the path to return
fn:doc("/content/article/test.xml")/ARTICLE/AUTHOR/FIRSTNAME but in
fact it stops on step upper.
But there is inconsistency here because if my firstname was a composed one (2
words), then the returned base is just not the same :
Toussaint
Jean Pierre
http://marklogic.com/appservices/search";>
Jean
Pierre
[...]
Is it a bug or can someone explain me why search result print different
information base on word count from element ?
By the way there is in fact no need of constraints to reproduce this exemple a
simple full text search provide the very same result.
I'd like to provide a filter step based on the path information, and this issue
did't allow me to do this.
Best Regards
Stephane
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
