[MarkLogic Dev General] Thesaurus question
Hi, Will MarkLogic have search issues if I leave the part-of-speech element out of my thsr doc? It's not required by the schema, but it appears in all of your examples. I will replicate many entries and that info will be unknown. Maybe it's added value but unnecessary? Thanks, Shannon (shifl...@virginia.edu) ___ General mailing list General@developer.marklogic.com http://community.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Marklogic Webdav not connecting
Thanks, JB.
I will give that a try!
My OS is Windows Server 2003 R2 64bit
-Original Message-
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Joseph Bryan
Sent: Wednesday, June 27, 2012 12:47 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Marklogic Webdav not connecting
What OS are you running?
I've had trouble on Windows 7 browsing the root of a WebDAV app
server. When I instead accessed a specific sub-directory via WebDAV, I
was able to view the contents in Windows Explorer.
Thanks.
-jb
On Wed, Jun 27, 2012 at 1:16 PM, DJaun Maclin wrote:
> Thanks for the suggestions, Danny!
>
> My Webdav was set to root and the root properties existed after testing in CQ
> Tried different users of varying permissions, but it didn't get me anywhere.
> Thanks though.
>
>
> -Original Message-
> From: general-boun...@developer.marklogic.com
> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Danny Sokolsky
> Sent: Wednesday, June 27, 2012 1:12 AM
> To: MarkLogic Developer Discussion
> Subject: Re: [MarkLogic Dev General] Marklogic Webdav not connecting
>
> Here are a few other things to look at:
>
> * Make sure the root directory exists in the database. For example, if the
> root is / on the WebDAV server, make sure the directory / exists (
> xdmp:directory-properties("/") should return a properties document with a
> directory element, for example)
> * make sure the user you are logging into the WebDAV server has permissions
> on that root directory. Try logging in as a user with the admin role--=if
> that works but another user does not, then the user probably does not have
> permissions on the root directory.
>
> -Danny
>
> From: general-boun...@developer.marklogic.com
> [general-boun...@developer.marklogic.com] On Behalf Of DJaun Maclin
> [dmac...@wattnet.net]
> Sent: Tuesday, June 26, 2012 11:12 PM
> To: General Discussion
> Cc: Robert Tuten; Matt Leshinskie
> Subject: Re: [MarkLogic Dev General] Marklogic Webdav not connecting
>
> Thanks for the reply, Danny!
> Automatic directory creation is enabled for the database.
>
> From: Danny Sokolsky
> mailto:danny.sokol...@marklogic.com>>
> Reply-To: General Discussion
> mailto:general@developer.marklogic.com>>
> Date: Tue, 26 Jun 2012 16:12:05 -0700
> To: General Discussion
> mailto:general@developer.marklogic.com>>
> Cc: Robert Tuten mailto:rtu...@wattnet.net>>, Matt
> Leshinskie
> mailto:matt.leshins...@marklogic.com>>
> Subject: Re: [MarkLogic Dev General] Marklogic Webdav not connecting
>
> Is automatic directory creation enabled for the database?
> ___
> General mailing list
> General@developer.marklogic.com
> http://community.marklogic.com/mailman/listinfo/general
> ___
> General mailing list
> General@developer.marklogic.com
> http://community.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://community.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://community.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Marklogic Webdav not connecting
Try connecting with BitKinex if you are using win-7. Also you can try
changing webdav server's default authentication configuration to "basic" if
not set already.
On Wed, Jun 27, 2012 at 1:47 PM, Joseph Bryan wrote:
> What OS are you running?
>
> I've had trouble on Windows 7 browsing the root of a WebDAV app
> server. When I instead accessed a specific sub-directory via WebDAV, I
> was able to view the contents in Windows Explorer.
>
> Thanks.
>
> -jb
>
> On Wed, Jun 27, 2012 at 1:16 PM, DJaun Maclin wrote:
> > Thanks for the suggestions, Danny!
> >
> > My Webdav was set to root and the root properties existed after testing
> in CQ
> > Tried different users of varying permissions, but it didn't get me
> anywhere.
> > Thanks though.
> >
> >
> > -Original Message-
> > From: general-boun...@developer.marklogic.com [mailto:
> general-boun...@developer.marklogic.com] On Behalf Of Danny Sokolsky
> > Sent: Wednesday, June 27, 2012 1:12 AM
> > To: MarkLogic Developer Discussion
> > Subject: Re: [MarkLogic Dev General] Marklogic Webdav not connecting
> >
> > Here are a few other things to look at:
> >
> > * Make sure the root directory exists in the database. For example, if
> the root is / on the WebDAV server, make sure the directory / exists (
> xdmp:directory-properties("/") should return a properties document with a
> directory element, for example)
> > * make sure the user you are logging into the WebDAV server has
> permissions on that root directory. Try logging in as a user with the
> admin role--=if that works but another user does not, then the user
> probably does not have permissions on the root directory.
> >
> > -Danny
> >
> > From: general-boun...@developer.marklogic.com [
> general-boun...@developer.marklogic.com] On Behalf Of DJaun Maclin [
> dmac...@wattnet.net]
> > Sent: Tuesday, June 26, 2012 11:12 PM
> > To: General Discussion
> > Cc: Robert Tuten; Matt Leshinskie
> > Subject: Re: [MarkLogic Dev General] Marklogic Webdav not connecting
> >
> > Thanks for the reply, Danny!
> > Automatic directory creation is enabled for the database.
> >
> > From: Danny Sokolsky danny.sokol...@marklogic.com>>
> > Reply-To: General Discussion general@developer.marklogic.com>>
> > Date: Tue, 26 Jun 2012 16:12:05 -0700
> > To: General Discussion general@developer.marklogic.com>>
> > Cc: Robert Tuten mailto:rtu...@wattnet.net>>, Matt
> Leshinskie matt.leshins...@marklogic.com>>
> > Subject: Re: [MarkLogic Dev General] Marklogic Webdav not connecting
> >
> > Is automatic directory creation enabled for the database?
> > ___
> > General mailing list
> > General@developer.marklogic.com
> > http://community.marklogic.com/mailman/listinfo/general
> > ___
> > General mailing list
> > General@developer.marklogic.com
> > http://community.marklogic.com/mailman/listinfo/general
> ___
> General mailing list
> General@developer.marklogic.com
> http://community.marklogic.com/mailman/listinfo/general
>
___
General mailing list
General@developer.marklogic.com
http://community.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Database Status Troubles
Thanks that's a big help, Michael On 06/27/2012 12:31 PM, Michael Blakeley wrote: > You can collect good data manually with something like 'iostat -mxz 15' > (remove the 'z' or the 'm' if not supported, but the 'x' is important). > > Or you can configure your sysstat cron job to collect disk statistics > automatically, for sar. In /etc/cron.d/sysstat you want something like this: > > >> */10 * * * * root /usr/lib64/sa/sa1 -S DISK 60 10& >> >> 53 23 * * * root /usr/lib64/sa/sa2 –A >> > The "-S DISK" is important, and isn't there by default. > > Also make sure sysstat is installed, of course. It includes iostat too. > > -- Mike > > On 27 Jun 2012, at 07:11 , Mike Sokolov wrote: > > >> We will need to trigger a reindex in the next couple of days, so I will >> capture some stats then and post back - thanks. >> >> -Mike >> >> On 06/27/2012 09:05 AM, Damon Feldman wrote: >> >>> Mike and Alex, >>> >>> MarkLogic can do both at the same time, but an underlying system issue may >>> be triggered by the intensified activity. I agree with a previous message >>> suggesting the OS is overloaded, swapping or otherwise in serious trouble. >>> The system should re-index without affecting queries very much, >>> particularly if the reindex throttle is turned down. >>> >>> Can you check the OS and/or VM status for CPU, swap and I/O activity? >>> >>> Yours, >>> Damon >>> >>> -Original Message- >>> From: general-boun...@developer.marklogic.com >>> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Mike Sokolov >>> Sent: Wednesday, June 27, 2012 7:52 AM >>> To: MarkLogic Developer Discussion >>> Cc: Danny Sokolsky >>> Subject: Re: [MarkLogic Dev General] Database Status Troubles >>> >>> I would agree that you are bumping up against hardware limits. >>> >>> I get this regularly on a largish database (> 10M docs, ~ 680 GB on one >>> server). Whenever we re-index a significant amount of content the >>> server becomes unusable for days - all it can handle is the reindexing >>> and merging. Even if we throttle reindexing down to 1 there will be >>> periods of downtime (and of course the whole process takes longer). The >>> admin console responds fine except for the status screens, which >>> (mostly) time out, as Alex reports. If you keep trying, you can usually >>> eventually get a response, although this can take 1/2 hour of retrying >>> and waiting, which is painful. >>> >>> The strategy we are pursuing is to maintain two servers; we reindex them >>> in sequence, switching production from one to the other and back again. >>> Another thing we are trying is increasing RAM (because we can, and the >>> system doesn't *seem* to be CPU bound), but I suspect the real issue is >>> disk I/O. >>> >>> I'd like to add more disk, or faster ones (or ideally more servers, but >>> that is challenging for a variety of reasons). I'm also not sure how to >>> prove that more I/O bandwidth would help dramatically. I've looked at >>> vmstat: we're not swapping, but I'm also not sure how to read the stats >>> clearly enough to understand in detail what's going on with the disks. >>> >>> Also wondering if you have recommendations as to how to configure >>> disks. Do you use RAID? What kind? A centralized SAN or other storage >>> device? I plan to ask support for help, too, but any general >>> suggestions from folks on the list here would be appreciated. >>> >>> -Mike >>> >>> On 6/26/2012 6:16 PM, Danny Sokolsky wrote: >>> >>> Another thing you might try is turning the reindexer throttle down. Go to the database config page and try setting reindexer throttle to 1 (I think the default is 5). Then give your system a while to catch up, and see if you can then get to the forest status page. -Danny -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Michael Blakeley Sent: Tuesday, June 26, 2012 2:37 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Database Status Troubles On 26 Jun 2012, at 14:12 , Alex Milowski wrote: > On Tue, Jun 26, 2012 at 2:01 PM, Michael Blakeley > wrote: > > >> On 26 Jun 2012, at 13:52 , Alex Milowski wrote: >> >> >> >>> The forest status page doesn't return. The browser just hangs out >>> until something times out. >>> >>> >> That is decidedly abnormal. I have seen a few systems where >> database-status was slow and would time out during reindexing, but >> forest-status should not. Admittedly I don't do much with geospatial, so >> this might be a geospatial-specific bug. >> >> > Here's what I don't get. I dropped a large index. Shouldn't that be > an easy operation? > > Or is it the index I added th
Re: [MarkLogic Dev General] Marklogic Webdav not connecting
What OS are you running?
I've had trouble on Windows 7 browsing the root of a WebDAV app
server. When I instead accessed a specific sub-directory via WebDAV, I
was able to view the contents in Windows Explorer.
Thanks.
-jb
On Wed, Jun 27, 2012 at 1:16 PM, DJaun Maclin wrote:
> Thanks for the suggestions, Danny!
>
> My Webdav was set to root and the root properties existed after testing in CQ
> Tried different users of varying permissions, but it didn't get me anywhere.
> Thanks though.
>
>
> -Original Message-
> From: general-boun...@developer.marklogic.com
> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Danny Sokolsky
> Sent: Wednesday, June 27, 2012 1:12 AM
> To: MarkLogic Developer Discussion
> Subject: Re: [MarkLogic Dev General] Marklogic Webdav not connecting
>
> Here are a few other things to look at:
>
> * Make sure the root directory exists in the database. For example, if the
> root is / on the WebDAV server, make sure the directory / exists (
> xdmp:directory-properties("/") should return a properties document with a
> directory element, for example)
> * make sure the user you are logging into the WebDAV server has permissions
> on that root directory. Try logging in as a user with the admin role--=if
> that works but another user does not, then the user probably does not have
> permissions on the root directory.
>
> -Danny
>
> From: general-boun...@developer.marklogic.com
> [general-boun...@developer.marklogic.com] On Behalf Of DJaun Maclin
> [dmac...@wattnet.net]
> Sent: Tuesday, June 26, 2012 11:12 PM
> To: General Discussion
> Cc: Robert Tuten; Matt Leshinskie
> Subject: Re: [MarkLogic Dev General] Marklogic Webdav not connecting
>
> Thanks for the reply, Danny!
> Automatic directory creation is enabled for the database.
>
> From: Danny Sokolsky
> mailto:danny.sokol...@marklogic.com>>
> Reply-To: General Discussion
> mailto:general@developer.marklogic.com>>
> Date: Tue, 26 Jun 2012 16:12:05 -0700
> To: General Discussion
> mailto:general@developer.marklogic.com>>
> Cc: Robert Tuten mailto:rtu...@wattnet.net>>, Matt
> Leshinskie
> mailto:matt.leshins...@marklogic.com>>
> Subject: Re: [MarkLogic Dev General] Marklogic Webdav not connecting
>
> Is automatic directory creation enabled for the database?
> ___
> General mailing list
> General@developer.marklogic.com
> http://community.marklogic.com/mailman/listinfo/general
> ___
> General mailing list
> General@developer.marklogic.com
> http://community.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://community.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] cts:search / geospatial queries - combining criteria
On Wed, Jun 27, 2012 at 10:21 AM, Michael Blakeley wrote:
> I'm not sure that was a good change. Sometimes more documents are better -
> and MarkLogic can handle billions, albeit with the right hardware.
>
> Anyway, look into fetching the station ids using cts:element-attribute-values
> instead of cts:search. You might also be able to move the received date-range
> constraints into a cts:query term on that call, which would allow you to use
> cts:frequency instead of xdmp:estimate.
>
I'm not sure it was a great move but having to index millions of
weather reports just to get one report working better doesn't feel
like a great solution either.
Just as another example, this one performs reasonably well for one quadrangle:
for $s in cts:search(
collection("http://.../stations/";),
cts:element-attribute-pair-geospatial-query(xs:QName("s:station"),
QName("","latitude"), QName("","longitude"), $quad) )
order by $s/s:station/@id
return if
(not(collection(concat("http://.../weather/",$s/s:station/@id))/s:report[@received>$dtstart
and @received<$dtend])[1]) then () else
let $id := string($s/s:station/@id)
...
even with the order by and testing for no reports.
I've also looks at the query plans for some of these. There is a
index on @received and so the expression [@received>$dtstart and
@received<$dtend] hits the index quite nicely. It only becomes
"faster" if you can combine it within one query (via an and) as I had
originally.
There's a twist here in that the report I'm having trouble with is for
all quadrangles where the above example is only for one quadrangles.
That is, in my original question, that query runs many times (10368
times for 2.5° quadrangles). In relational terms, it is a join that
I'm performing multiple times because I'm grouping by quadrangle.
--
--Alex Milowski
"The excellence of grammar as a guide is proportional to the paucity of the
inflexions, i.e. to the degree of analysis effected by the language
considered."
Bertrand Russell in a footnote of Principles of Mathematics
___
General mailing list
General@developer.marklogic.com
http://community.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Wikimedia parse
Ah the API does !!! Who hoo.
Maybe I can get XML out of this after all ... I smell an xmlsh extension in the
making :)
I actually have a similar problem with xmlsh docs ... they are all currently in
WakiWiki ... but thats a black box ... I want to turn them into XML like
DocBook ...
Someone (Dave Pawson I think ?) wrote me a python lib to do that but only 90%
... the last 10% as usual is 99%
-
David Lee
Lead Engineer
MarkLogic Corporation
d...@marklogic.com
Phone: +1 650-287-2531
Cell: +1 812-630-7622
www.marklogic.com
This e-mail and any accompanying attachments are confidential. The information
is intended solely for the use of the individual to whom it is addressed. Any
review, disclosure, copying, distribution, or use of this e-mail communication
by others is strictly prohibited. If you are not the intended recipient, please
notify us immediately by returning this message to the sender and delete all
copies. Thank you for your cooperation.
> -Original Message-
> From: general-boun...@developer.marklogic.com [mailto:general-
> boun...@developer.marklogic.com] On Behalf Of Michael Blakeley
> Sent: Wednesday, June 27, 2012 1:23 PM
> To: MarkLogic Developer Discussion
> Subject: Re: [MarkLogic Dev General] Wikimedia parse
>
> Yes, it does. The API gives you access to much of that wiki-structured markup,
> but you have to decide what to do with it. Naturally the online tool and much
> of the sample code doesn't do anything interesting.
>
> -- Mike
>
> On 27 Jun 2012, at 10:19 , David Lee wrote:
>
> > Thanks, I tried the online tool on a sample I have and it strips out much
> > of
> the meaningful stuff :(
> >
> > --- Input
> > {{Infobox settlement
> >
> >
> > |image_skyline =
> > |imagesize =
> > |image_caption =
> > |image_flag =
> > |flag_size =
> > |image_seal =
> > |seal_size =
> > |image_shield =
> > |shield_size=
> > |image_map =
> > |mapsize=
> > |map_caption=
> > |pushpin_map=Mali
> > |pushpin_label_position =bottom
> > |pushpin_mapsize=300
> > |pushpin_map_caption=Location in Mali
> >
> > |coordinates_display= inline,title
> > |coordinates_region = ML
> > |subdivision_type = Country
> > |subdivision_name = {{flag|Mali}}
> > |subdivision_type1 = [[Regions of Mali|Region]]
> > |subdivision_name1 = [[Kayes Region]]
> > |subdivision_type2 =[[Cercles of Mali|Cercle]]
> > |subdivision_name2 = [[Kayes Cercle]]
> > |subdivision_type3 =[[Communes of Mali|Commune]]
> > |subdivision_name3 = [[Karakoro]]
> > |
> > |leader_name1 =
> > |established_title =
> > |established_date =
> >
> > |area_footnotes =
> > |area_total_km2 =
> > |area_land_km2=
> >
> > |population_as_of =
> > |population_footnotes =
> > |population_note=
> > |population_total =
> > |population_density_km2 =
> > |population_density_sq_mi =
> > |population_metro =
> > |population_density_metro_km2 =
> > |population_density_metro_sq_mi =
> > |population_blank1_title=Ethnicities
> > |population_blank1 =
> > |population_density_blank1_km2 =
> > |population_density_blank1_sq_mi =
> >
> > |timezone =[[GMT]]
> > |utc_offset = +0
> > |timezone_DST =
> > |utc_offset_DST =
> > |latd=15|latm=16|lats=30 |latNS=N
> > |longd=11|longm=42|longs=25|longEW=W
> > |elevation_footnotes=
>
> |nickname =
> |settlement_type=Village
> |motto =
>
> |image_skyline =
> |imagesize =
> |image_caption =
> |image_flag =
> |flag_size =
> |image_seal =
> |seal_size =
> |image_shield =
> |shield_size=
> |image_map =
> |mapsize=
> |map_caption=
> |pushpin_map=Mali
> |pushpin_label_position =bottom
> |pushpin_mapsize=300
> |pushpin_map_caption=Location in Mali
>
> |coordinates_display= inline,title
> |coordinates_region = ML
> |subdivision_type = Country
> |subdivision_name = {{flag|Mali}}
> |subdivision_type1 = [[Regions of Mali|Region]]
> |subdivision_name1 = [[Kayes Region]]
> |subdivision_type2 =[[Cercles of Mali|Cercle]]
> |subdivision_name2 = [[Kayes Cercle]]
> |subdivision_type3 =[[Communes of Mali|Commune]]
> |subdivision_name3 = [[Karakoro]]
> |
> |leader_name1 =
> |established_title =
> |established_date =
>
> |area_footnotes =
> |area_total_km2 =
> |area_land_km2=
>
> |population_as_of =
> |population_footnotes =
> |population_note=
> |population_total =
> |population_density_km2 =
> |population_density_sq_mi =
> |population_metro =
> |population_density_metro_km2 =
> |population_density_metro_sq_mi =
> |population_blank1_title=Ethnicities
> |population_blank1 =
> |population_density_blank1_km2 =
> |population_density_blank1_sq_mi =
>
> |timezone =[[GMT]]
> |utc_offset = +0
> |timezone_DST =
> |utc_offset_DST =
> |latd=15|latm=16|lats=30 |latNS=N
> |longd=11|longm=42|longs=25|longEW=W
> |elevation_footnotes=
> |elevation_m=
> |elevation_ft =
>
> |postal_code=
> |area_code =
> |blank_name =
> |blank_info =
> |blank1_name=
> |blank1_info=
> |website=
> |footnotes =
> }}
>
> '''Teichibe''' is a village and principal settlement (''[[chef-lieu]]'') of
> the [[Karakoro|commune of Karakoro]] in the [[Kayes Cercle|Cercle of Kayes]]
> in the [[Kayes Region]] of south-western [[Mali]].{{citation |
> title=Communes de la Région de Kayes | publisher= Ministère de
> l'administration territoriale et des collectivités locales, République du
> Mali | url=http://www.matcl.gov.ml/pdf/ComRegKayes.pdf | language=French
> }}.
>
>
> ==References==
> {{reflist}}
>
> [[Category:Populated places in the Kayes Region]]
>
>
> {{Kayes-geo-stub}}
>
> -- Output
>
> {{Infobox settlement
>
>
|nickname =
|settlement_type=Village
|motto =
|image_skyline =
|imagesize =
|image_caption =
|image_flag =
|flag_size =
|image_seal =
|seal_size =
|image_shield =
|shield_size=
|image_map =
|mapsize=
|map_caption=
|pushpin_map=Mali
|pushpin_label_position =bottom
|pushpin_mapsize=300
|pushpin_map_caption=Location in Mali
|coordinates_display= inline,title
|coordinates_region = ML
|subdivision_type = Country
|subdivision_name = {{flag|Mali}}
|subdivision_type1 = [[Regions of Mali|Region]]
|subdivision_name1 = [[Kayes Region]]
|subdivision_type2 =[[Cercles of Mali|Cercle]]
|subdivision_name2 = [[Kayes Cercle]]
|subdivision_type3 =[[Communes of Mali|Commune]]
|subdivision_name3 = [[Karakoro]]
|
|leader_name1 =
|established_title =
|established_date =
|area_footnotes =
|area_total_km2 =
|area_land_km2=
|population_as_of =
|population_footnotes =
|population_note=
|population_total =
|population_density_km2 =
|population_density_sq_mi =
|population_metro =
|population_density_metro_km2 =
|population_density_metro_sq_mi =
|population_blank1_title=Ethnicities
|population_blank1 =
|population_density_blank1_km2 =
|population_density_blank1_sq_mi =
|timezone =[[GMT]]
|utc_offset = +0
|timezone_DST =
|utc_offset_DST =
|latd=15|latm=16|lats=30 |latNS=N
|longd=11|longm=42|longs=25|longEW=W
|elevation_footnotes=
|elevation_m=
|elevation_ft =
|postal_code=
|area_code =
|blank_name =
|blank_info =
|blank1_name=
|blank1_info=
|website=
|footnotes =
}}
'''Teichibe''' is a village and principal settlement (''[[chef-lieu]]'') of the
[[Karakoro|commune of Karakoro]] in the [[Kayes Cercle|Cercle of Kayes]] in the
[[Kayes Region]] of south-western [[Mali]].{{citation |
title=Communes de la Région de Kayes | publisher= Ministère de l'administration
territoriale et des collectivités locales, République du Mali |
url=http://www.matcl.gov.ml/pdf/ComRegKayes.pdf | language=French
}}.
==References==
{{reflist}}
[[Category:Populated places in the Kayes Region]]
{{Kayes-geo-stub}}
-- Output
{{Infobox settlement