[MarkLogic Dev General] MarkLogic 5 Academic License is now available

2012-03-26 Thread Christie Hoyle 
Dear MarkLogic Community,

In an effort to increase participation in the MarkLogic Community, we will 
announce a new Academic License  for students and educators on March 27.  The 
goal of the Academic License is to engage the academic community, educate them 
about MarkLogic, and ultimately enable a new wave of MarkLogic developers.  We 
are excited to bring our enterprise grade technology to academia with features 
that will enable academics to realize the ease of use, flexibility, and power 
of MarkLogic.  This will enable students to gain real experience using 
MarkLogic for Big Data problems and take that experience into the marketplace 
through research, projects, or paid employment.

The Academic License, available for MarkLogic 5, is free to faculty members or 
students who join the MarkLogic community.  The Academic License provides 
features available within the Express license, such as geospatial, alerting, 
and conversion, as well as features otherwise available only in Enterprise 
Edition, like clustering and replication.  It has a 2-year time limit and 
cannot be used for commercial or government projects.  After MarkLogic 5 is 
downloaded, installed, and running, a user can browse to the admin interface 
and request an Academic license directly from within the interface.

Free MarkLogic Academic and Express Licenses are subject to the terms and 
conditions of the online software academic and express license agreements that 
accompany each download.  Please contact the Community Team with any questions.

Kind Regards,

MarkLogic Community Team
Christie Hoyle, Eric Bloch and Evan Lenz


Christie Hoyle
Community Marketing Manager
MarkLogic Corporation
christie.ho...@marklogic.com
Phone: +1 650 287 3510
Cell:  +1 415 971 1807
www.marklogic.com

[cid:image001.png@01CD0B96.1E90D2E0]

This e-mail and any accompanying attachments are confidential. The information 
is intended solely for the use of the individual to whom it is addressed. Any 
review, disclosure, copying, distribution, or use of this e-mail communication 
by others is strictly prohibited. If you are not the intended recipient, please 
notify us immediately by returning this message to the sender and delete all 
copies. Thank you for your cooperation.

<>___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Corona Questions

2012-03-26 Thread Randy Smith 

Eric & Others,
After reading more I though you could be correct that all documents 
would have to be loaded with Corona. So I tried to load a document using 
the example commands but that didn't work. Kept getting a 404. Maybe the 
URI path I'm giving it is an issue (e.g. 
http://myhost:8066/store?uri=c:/documents/myfile.xml). After reading 
even more I'm thinking one should be able to load the documents with 
normal methods like Info Studio( like I did earlier). I've installed 
Corona on three different machines (i.e. Mac, Bootcamp with XP, and a 
another XP client). The results seem to be about the same as I've 
indicated here.


Corona sounds like just what I need but can't seem to find why it 
doesn't work. ugh!

Thanks for the help!
Randy

On 3/23/2012 11:41 AM, Eric Bloch wrote:
I know at one point Corona only knew about docs inserted via Corona, 
but I believe that behavior has changed.  Hm…


E




On Mar 23, 2012, at 9:25 AM, Randy Smith wrote:


Geert,
I ran the CQ search you suggested and lots of results came back for 
'cat'. I am puzzled why it is not working. I even created a new 
version of the top-songs database which worked fine with CQ but with 
no results when running the Corona test.


I looked over the installation but did not see any problem. I wonder 
if there could be a permission problem. Are the two Corona accounts 
that were created when Corona was setup involved when running Corona? 
I am currently running as admin when I run my tests.

Thanks for the help so far.
Ideas?
Randy

On 3/23/2012 1:36 AM, Geert Josten wrote:


Hi Randy,


The reason I asked is because you aren’t getting back results. That 
sounds either that there really aren’t any docs containing the word 
‘cat’ (less likely I thought), or you are running against an empty 
database. Did you double check? You could also run a test query in 
Query Console (or CQ if you are using ML 4) to make sure, with 
something like:



cts:search(collection(), ‘cat’)


(and selecting the correct database in the console of course..)


Kind regards,

Geert


*Van:*general-boun...@developer.marklogic.com 
 
[mailto:general-boun...@developer.marklogic.com 
] *Namens *Randy Smith

*Verzonden:* donderdag 22 maart 2012 22:38
*Aan:* MarkLogic Developer Discussion
*Onderwerp:* Re: [MarkLogic Dev General] Corona Questions


Geert,
Eric is correct, I point it to the top-songs database we used in 
class. My app we did in class works against the same database so it 
should be ok. You said something about the config of Corona. I have 
not touched that. Is that a place to look for issues?

Randy

On 3/22/2012 4:18 PM, Geert Josten wrote:

Hi Randy,


Are you sure corona is running against the correct documents 
database? Your rest call works just fine with me, returning proper 
results..



Kind regards,

Geert


*Van:*general-boun...@developer.marklogic.com 
 
[mailto:general-boun...@developer.marklogic.com 
] *Namens *Randy Smith

*Verzonden:* donderdag 22 maart 2012 22:01
*Aan:* MarkLogic Developer Discussion
*Onderwerp:* Re: [MarkLogic Dev General] Corona Questions


Geert,
I'm running the latest copy (a few days old).

 I just found that when I do 
|http//myhost:8011/search?stringQuery=cat&outputFormat=xml | I now 
get back an xml file in the search results format  indicated in the 
documentation. That has got to be a good sign!!! :-) The only 
problem is that the "results" tag is empty. The meta tag also comes 
back but has all zeros.

This is the xml file as it comes back in the browser:
*


0
0
0*
*0S*
*

*

Note that the executionTime comes back with something like .01 if I 
leave off the "outputFormat=xml. So, that works if it thinks it is JSON.


Hope this offers some clues!
Thanks for the help!
Randy
On 3/22/2012 3:23 PM, Geert Josten wrote:

Hi Randy,


A 404 likely means the endpoints config is wonky. Are you running 
latest copy from git? The 500 indicates something goes wrong 
internally. Can you tell what response you are getting exactly? In 
case of a 500, there should be more details in the response body.



Kind regards,

Geert


*Van:*general-boun...@developer.marklogic.com 
 
[mailto:general-boun...@developer.mar 
klogic.com 
] *Namens *Randy Smith

*Verzonden:* donderdag 22 maart 2012 20:17
*Aan:* General MarkLogic Developer Discussion
*Onderwerp:* [MarkLogic Dev General] Corona Questions


All,
Trying to come up to speed with the Alpha version of Corona. i.e. 
the REST connection to MarkLogic.

*BTW this is what I really need so thanks to the developer group!*

I want to use it with a .NET app, i.e. send http queries, requests, 
etc and use the MarkLogic results in the .NET app.

Firs,t this is my view of how Corona works which ma

Re: [MarkLogic Dev General] Internal Server Error

2012-03-26 Thread Harry B. 
CQ is now Query Console and is available at port 8000/qconsole

It is wildly improved. There's a good document on the community site.
On Mar 26, 2012 11:18 AM, "Khan, Kashif"  wrote:

> I will try this but one more thing that I noticed is that CQ has also
> stopped working.
>
>
> Best Regards,
> Kashif Khan
>
>
>
>
> On 3/26/12 1:11 PM, "Michael Blakeley"  wrote:
>
> >Try adding some whitespace before the src attribute, as required by
> >http://www.w3.org/TR/xquery/#prod-xquery-DirAttributeList - whitespace is
> >optional around the '=' but required before each QName.
> >
> >If it worked before, I believe that was a bug. If so, the fix was
> >probably part of the unquoted attribute value fix.
> >
> >-- Mike
> >
> >On 26 Mar 2012, at 09:56 , Khan, Kashif wrote:
> >
> >>  http://developer.marklogic.com/mailman/listinfo/general
>
> ___
> General mailing list
> General@developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general
>
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Internal Server Error

2012-03-26 Thread Khan, Kashif 
When I try to load the file footer.xqy it is throwing error in my WebDav
but it still gets loaded. When I try to download it the file is empty all
the code vanished from the file. Really strange.



Best Regards,
Kashif Khan
Sr. Solutions Architect
Houghton Mifflin Harcourt, Orlando, FL
Office: (407) 345-3420
Cell: (407) 949-4697
 

"The water you touch in the river is the last of that which has passed and
the first of that which is coming" --Leonardo da Vinci




On 3/26/12 1:18 PM, "Khan, Kashif"  wrote:

>I will try this but one more thing that I noticed is that CQ has also
>stopped working.
>
>
>Best Regards,
>Kashif Khan
>
>
>
>
>On 3/26/12 1:11 PM, "Michael Blakeley"  wrote:
>
>>Try adding some whitespace before the src attribute, as required by
>>http://www.w3.org/TR/xquery/#prod-xquery-DirAttributeList - whitespace is
>>optional around the '=' but required before each QName.
>>
>>If it worked before, I believe that was a bug. If so, the fix was
>>probably part of the unquoted attribute value fix.
>>
>>-- Mike
>>
>>On 26 Mar 2012, at 09:56 , Khan, Kashif wrote:
>>
>>> http://developer.marklogic.com/mailman/listinfo/general
>
>___
>General mailing list
>General@developer.marklogic.com
>http://developer.marklogic.com/mailman/listinfo/general

___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Internal Server Error

2012-03-26 Thread Khan, Kashif 
I will try this but one more thing that I noticed is that CQ has also
stopped working.


Best Regards,
Kashif Khan




On 3/26/12 1:11 PM, "Michael Blakeley"  wrote:

>Try adding some whitespace before the src attribute, as required by
>http://www.w3.org/TR/xquery/#prod-xquery-DirAttributeList - whitespace is
>optional around the '=' but required before each QName.
>
>If it worked before, I believe that was a bug. If so, the fix was
>probably part of the unquoted attribute value fix.
>
>-- Mike
>
>On 26 Mar 2012, at 09:56 , Khan, Kashif wrote:
>
>>  http://developer.marklogic.com/mailman/listinfo/general

___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Internal Server Error

2012-03-26 Thread Michael Blakeley 
Try adding some whitespace before the src attribute, as required by 
http://www.w3.org/TR/xquery/#prod-xquery-DirAttributeList - whitespace is 
optional around the '=' but required before each QName.

If it worked before, I believe that was a bug. If so, the fix was probably part 
of the unquoted attribute value fix.

-- Mike

On 26 Mar 2012, at 09:56 , Khan, Kashif wrote:

>   http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Internal Server Error

2012-03-26 Thread David Sewell 
Speaking of bug fixes, the release notes at support.marklogic.com haven't yet 
been updated to reflect fixes in 5.0-3.

On Mon, 26 Mar 2012, John Snelson wrote:

> What does the code in footer.xqy look like?
>
> One bug fix that was made in 5.0-1 was to require quotes around
> attribute values in direct element constructor expressions - maybe you
> have unquoted attribute values in your program?
>
> John
>
> On 26/03/12 17:22, Khan, Kashif wrote:
>>
>>   We just upgraded to Marklogic server 5.0-3 and now I am getting this
>>   error for my application. Did anyone face this issue with their
>>   application?
>>
>>
>>   500 Internal Server Error
>>
>> XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error,
>> unexpected QName_, expecting EmptyTagClose_ or StartTagClose_
>> in /application/views/footer.xqy, at 16:31,
>> in xdmp:invoke("/application/views/footer.xqy", (fn:QName("", "data"),
>> map:map()), > xmlns="xdmp:eval">different-transactiont...)
>> [1.0-ml]
>> in /system/xqmvc.xqy, at 105:4,
>> in _view("/application/views/footer.xqy", ()) [1.0-ml]
>> $view-file = "/application/views/footer.xqy"
>> $pairs = ()
>> in /system/xqmvc.xqy, at 119:8,
>> in xqmvc:view("footer", ()) [1.0-ml]
>> $view = "footer"
>> $pairs = ()
>> $view-file = "/application/views/footer.xqy"
>> in /application/controllers/word.xqy, at 216:12,
>> in word-controller:error() [1.0-ml]
>> in /system/error.xqy, at 9:7 [1.0-ml]
>> $set-var = ()
>> Old Error
>> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">;
>> XDMP-UNEXPECTED
>> err:XPST0003
>> 1.0-ml
>> Unexpected token
>> XDMP-UNEXPECTED: (err:XPST0003)
>> Unexpected token syntax error, unexpected QName_, expecting
>> EmptyTagClose_ or StartTagClose_
>> false
>>   
>> syntax error, unexpected QName_, expecting
>> EmptyTagClose_ or StartTagClose_
>>   
>> 
>> /application/views/footer.xqy
>> 16
>> 31
>> xdmp:invoke("/application/views/footer.xqy",
>> (fn:QName("", "data"), map:map()), <options
>> 
>> xmlns="xdmp:eval"><isolation>different-transaction</isolation><prevent-deadlocks>t...</options>)
>> 1.0-ml
>>  
>> /system/xqmvc.xqy
>> 105
>> 4
>> _view("/application/views/footer.xqy",
>> ()) 
>>  > 
>> xmlns="http://scholarsportal.info/xqmvc/core">view-file;
>> "/application/views/footer.xqy"
>>   > xmlns="http://scholarsportal.info/xqmvc/core">pairs;
>> () 
>> 
>> 1.0-ml
>>  
>> /system/xqmvc.xqy
>> 119
>> 8
>> xqmvc:view("footer",
>> ()) 
>>  > xmlns="http://scholarsportal.info/xqmvc/core">view;
>> "footer"
>>   > xmlns="http://scholarsportal.info/xqmvc/core">pairs;
>> () 
>>  > 
>> xmlns="http://scholarsportal.info/xqmvc/core">view-file;
>> "/application/views/footer.xqy"
>>  
>> 1.0-ml
>>  
>> /application/controllers/word.xqy
>> 121
>> 18
>> search()
>>   > 
>> xmlns="http://scholarsportal.info/xqmvc/controller">id-query;
>> () 
>>  > 
>> xmlns="http://scholarsportal.info/xqmvc/controller">word-query;
>> () <

Re: [MarkLogic Dev General] Internal Server Error

2012-03-26 Thread Khan, Kashif 
Here is what is in the footer.xqy file

xquery version "1.0-ml";
import module namespace xqmvc = "http://scholarsportal.info/xqmvc/core"; at
"../../system/xqmvc.xqy";
declare variable $data as map:map external;





http://www.houghtonmifflinbooks.com";> 
Home
http://www.houghtonmifflinbooks.com/faq/";>|
FAQs
http://www.houghtonmifflinbooks.com/site_map/";>|
Sitemap







http://www.hmhco.com/privacy-policy.html";>Privacy
Policy  
http://www.houghtonmifflinbooks.com/trademark/";>|
Trademark Information
http://www.hmhco.com/terms-and-conditions.html";>| 
Terms
& Conditions of Use


For licensing inquiries, please contact
Damian Dale, mailto:damian.d...@hmhpub.com";>damian.d...@hmhpub.com
Copyright {(:© :)} 2011 Houghton Mifflin Harcourt 
Publishing
Company. All rights reserved.


   
   




Best Regards,
Kashif Khan




On 3/26/12 12:40 PM, "John Snelson"  wrote:

>What does the code in footer.xqy look like?
>
>One bug fix that was made in 5.0-1 was to require quotes around
>attribute values in direct element constructor expressions - maybe you
>have unquoted attribute values in your program?
>
>John
>
>On 26/03/12 17:22, Khan, Kashif wrote:
>>
>>   We just upgraded to Marklogic server 5.0-3 and now I am getting this
>>   error for my application. Did anyone face this issue with their
>>   application?
>>
>>
>>   500 Internal Server Error
>>
>> XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error,
>> unexpected QName_, expecting EmptyTagClose_ or StartTagClose_
>> in /application/views/footer.xqy, at 16:31,
>> in xdmp:invoke("/application/views/footer.xqy", (fn:QName("", "data"),
>> map:map()), > 
>>xmlns="xdmp:eval">different-transaction>adlocks>t...)
>> [1.0-ml]
>> in /system/xqmvc.xqy, at 105:4,
>> in _view("/application/views/footer.xqy", ()) [1.0-ml]
>> $view-file = "/application/views/footer.xqy"
>> $pairs = ()
>> in /system/xqmvc.xqy, at 119:8,
>> in xqmvc:view("footer", ()) [1.0-ml]
>> $view = "footer"
>> $pairs = ()
>> $view-file = "/application/views/footer.xqy"
>> in /application/controllers/word.xqy, at 216:12,
>> in word-controller:error() [1.0-ml]
>> in /system/error.xqy, at 9:7 [1.0-ml]
>> $set-var = ()
>> Old Error
>> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">;
>> XDMP-UNEXPECTED
>> err:XPST0003
>> 1.0-ml
>> Unexpected token
>> XDMP-UNEXPECTED: (err:XPST0003)
>> Unexpected token syntax error, unexpected QName_, expecting
>> EmptyTagClose_ or StartTagClose_
>> false
>>   
>> syntax error, unexpected QName_, expecting
>> EmptyTagClose_ or StartTagClose_
>>   
>> 
>> /application/views/footer.xqy
>> 16
>> 31
>> xdmp:invoke("/application/views/footer.xqy",
>> (fn:QName("", "data"), map:map()), <options
>> 
>>xmlns="xdmp:eval"><isolation>different-transaction&a
>>mp;lt;/isolation><prevent-deadlocks>t...</opt
>>ions>)
>> 1.0-ml
>>  
>> /system/xqmvc.xqy
>> 105
>> 4
>> _view("/application/views/footer.xqy",
>> ()) 
>>  > 
>>xmlns="http://scholarsportal.info/xqmvc/core">view-file>>
>> 
>>"/application/views/footer.xqy"
>>   > 
>>xmlns="http://scholarsportal.info/xqmvc/core">pairs;
>> () 
>> 
>> 1.0-ml
>>  
>> /system/xqmvc.xqy
>> 119
>> 8
>> xqmvc:view("footer",
>> ()) 
>>  > 
>>xmlns="http://scholarsportal.info/xqmvc/core">view;
>> "footer"
>>

Re: [MarkLogic Dev General] Internal Server Error

2012-03-26 Thread John Snelson 
What does the code in footer.xqy look like?

One bug fix that was made in 5.0-1 was to require quotes around 
attribute values in direct element constructor expressions - maybe you 
have unquoted attribute values in your program?

John

On 26/03/12 17:22, Khan, Kashif wrote:
>
>   We just upgraded to Marklogic server 5.0-3 and now I am getting this
>   error for my application. Did anyone face this issue with their
>   application?
>
>
>   500 Internal Server Error
>
> XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error,
> unexpected QName_, expecting EmptyTagClose_ or StartTagClose_
> in /application/views/footer.xqy, at 16:31,
> in xdmp:invoke("/application/views/footer.xqy", (fn:QName("", "data"),
> map:map()),  xmlns="xdmp:eval">different-transactiont...)
> [1.0-ml]
> in /system/xqmvc.xqy, at 105:4,
> in _view("/application/views/footer.xqy", ()) [1.0-ml]
> $view-file = "/application/views/footer.xqy"
> $pairs = ()
> in /system/xqmvc.xqy, at 119:8,
> in xqmvc:view("footer", ()) [1.0-ml]
> $view = "footer"
> $pairs = ()
> $view-file = "/application/views/footer.xqy"
> in /application/controllers/word.xqy, at 216:12,
> in word-controller:error() [1.0-ml]
> in /system/error.xqy, at 9:7 [1.0-ml]
> $set-var = ()
> Old Error
>  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">;
> XDMP-UNEXPECTED
> err:XPST0003
> 1.0-ml
> Unexpected token
> XDMP-UNEXPECTED: (err:XPST0003)
> Unexpected token syntax error, unexpected QName_, expecting
> EmptyTagClose_ or StartTagClose_
> false
>   
> syntax error, unexpected QName_, expecting
> EmptyTagClose_ or StartTagClose_
>   
> 
> /application/views/footer.xqy
> 16
> 31
> xdmp:invoke("/application/views/footer.xqy",
> (fn:QName("", "data"), map:map()), <options
> 
> xmlns="xdmp:eval"><isolation>different-transaction</isolation><prevent-deadlocks>t...</options>)
> 1.0-ml
>  
> /system/xqmvc.xqy
> 105
> 4
> _view("/application/views/footer.xqy",
> ()) 
>   
> xmlns="http://scholarsportal.info/xqmvc/core">view-file;
> "/application/views/footer.xqy"
>    xmlns="http://scholarsportal.info/xqmvc/core">pairs;
> () 
> 
> 1.0-ml
>  
> /system/xqmvc.xqy
> 119
> 8
> xqmvc:view("footer",
> ()) 
>   xmlns="http://scholarsportal.info/xqmvc/core">view;
> "footer"
>    xmlns="http://scholarsportal.info/xqmvc/core">pairs;
> () 
>   
> xmlns="http://scholarsportal.info/xqmvc/core">view-file;
> "/application/views/footer.xqy"
>  
> 1.0-ml
>  
> /application/controllers/word.xqy
> 121
> 18
> search()
>    
> xmlns="http://scholarsportal.info/xqmvc/controller">id-query;
> () 
>   
> xmlns="http://scholarsportal.info/xqmvc/controller">word-query;
> () 
>   
> xmlns="http://scholarsportal.info/xqmvc/controller">word-def;
> () 
>   
> xmlns="http://scholarsportal.info/xqmvc/

[MarkLogic Dev General] Internal Server Error

2012-03-26 Thread Khan, Kashif 
We just upgraded to Marklogic server 5.0-3 and now I am getting this error for 
my application. Did anyone face this issue with their application?
500 Internal Server Error

XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error, unexpected 
QName_, expecting EmptyTagClose_ or StartTagClose_
in /application/views/footer.xqy, at 16:31,
in xdmp:invoke("/application/views/footer.xqy", (fn:QName("", "data"), 
map:map()), different-transactiont...)
 [1.0-ml]
in /system/xqmvc.xqy, at 105:4,
in _view("/application/views/footer.xqy", ()) [1.0-ml]
$view-file = "/application/views/footer.xqy"
$pairs = ()
in /system/xqmvc.xqy, at 119:8,
in xqmvc:view("footer", ()) [1.0-ml]
$view = "footer"
$pairs = ()
$view-file = "/application/views/footer.xqy"
in /application/controllers/word.xqy, at 216:12,
in word-controller:error() [1.0-ml]
in /system/error.xqy, at 9:7 [1.0-ml]
$set-var = ()
Old Error

;
  XDMP-UNEXPECTED
  err:XPST0003
  1.0-ml
  Unexpected token
  XDMP-UNEXPECTED: (err:XPST0003) Unexpected
token syntax error, unexpected QName_, expecting EmptyTagClose_ or
StartTagClose_
  false
   
  
syntax error, unexpected QName_, expecting
EmptyTagClose_ or StartTagClose_

  
  

  /application/views/footer.xqy
  16
  31

xdmp:invoke("/application/views/footer.xqy",
(fn:QName("", "data"), map:map()), <options
xmlns="xdmp:eval"><isolation>different-transaction</isolation><prevent-deadlocks>t...</options>)

  1.0-ml


  /system/xqmvc.xqy
  105
  4
  _view("/application/views/footer.xqy",
())
  

  view-file;


"/application/views/footer.xqy"


  pairs;

  ()

  
  1.0-ml


  /system/xqmvc.xqy
  119
  8
  xqmvc:view("footer",
())
  

  view;
  "footer"


  pairs;

  ()


  view-file;


"/application/views/footer.xqy"

  
  1.0-ml



/application/controllers/word.xqy
  121
  18
  search()
  

  id-query;

  ()


  word-query;

  ()


  word-def;

  ()


  page-title;

  "American Heritage Dictionary -
Search"

  
  1.0-ml



/application/controllers/word.xqy

Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

2012-03-26 Thread Geert Josten 
Don’t think there is a lexicon for element names. Could make sense though.



I liked the map:map approach, so thought to give it a go. I have a database
with about 400k small docs. The following code ran in a few sec on the
first 1000 docs:



let $elems := map:map()

let $process-docs :=

  for $d in collection("feeds")[1 to 1000]

  for $elem in $d//*/local-name()

  where empty(map:get($elems, $elem))

  return

map:put($elems, $elem, 1)

for $elem in map:keys($elems)

order by $elem

return $elem,



xdmp:elapsed-time()



Subsequent run executes in quarter sec, thanks to caching.



Profile shows there are 54k elements in the first 1000 docs, of which just
25 unique.

First 2000 takes 7 sec, walks 100k elements, still 25 unique.

First 3000 still takes 7 sec, walks 160k elements, still just 25 unique.



(I have pretty much always the same doc structure, so that explains the
consistent 25 unique element names.)



First 10k takes 23,7 sec, 540k elems. (Still not maxing out!)

First 50k takes 2m40s, 2.6m elems, still works. (Seems to scale quite
linearly too!)

All: 410k takes about 19m, 64m expressions, 21m elems walked, still just 25
unique names. (PS: had to increased app server max time limit, which was
limited to 10 min)



Note: these times include profiling overhead.



So, it is doable, but not something you’d like to run each time,
particularly if the database size increases. Combining the above with a
post-commit trigger to keep a server-field maintained list up to date makes
most sense to me.



Kind regards,

Geert





*Van:* general-boun...@developer.marklogic.com [mailto:general-bounces@
developer.marklogic.com] *Namens *Brent Hartwig
*Verzonden:* maandag 26 maart 2012 15:45
*Aan:* sai shanker; MarkLogic Developer Discussion
*Onderwerp:* Re: [MarkLogic Dev General] Fwd: [1.0-ml]
XDMP-EXPNTREECACHEFULL



Curious there aren’t functions like this tapping into the universal index.



-Brent



*From:* general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] *On Behalf Of *sai shanker
*Sent:* Monday, March 26, 2012 9:23 AM
*To:* MarkLogic Developer Discussion
*Subject:* Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL





hi,

you can loop across each document, grab all the child nodes and put them in
a map.

Thanks and Regards,
Sai.





*From:* Ryan Dew 
*To:* MarkLogic Developer Discussion 
*Sent:* Monday, March 26, 2012 9:14 AM
*Subject:* Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL



You could try a recursive function like the following. No guarantee it is
100% right, if you have sub elements that have the same names as your root
elements.



xquery version "1.0-ml";



declare function local:find-unique-qnames($found-qnames as xs:QName*) {

  let $next-qname := cts:search(collection()/*,

if (exists($found-qnames))

then cts:not-query(cts:element-query($found-qnames,cts:and-query((

else cts:and-query(())

  )[1]/node-name(.)

  return if (exists($next-qname))

  then local:find-unique-qnames(($found-qnames,$next-qname))

  else $found-qnames

};



declare function local:find-unique-qnames() {

  for $qn in local:find-unique-qnames(())

  order by string($qn)

  return $qn

};



local:find-unique-qnames()



On Mon, Mar 26, 2012 at 6:36 AM, Geert Josten  wrote:

Hi Vishnu,



It would help if you could explain why you need that list. But in general
the best option would be to pre-calculate the list I guess. You can save it
as a server-field (xdmp:set-server-field), to keep the list in memory on
each host. But you would need an algorithm to initialize it, and each doc
commit would have to check and update that list. The latter can be done
with a post-commit trigger. The first can be done best by the strategy I
already mentioned: divide all docs in chunks of 100 to 1000 docs, calculate
distinct names of each chunk, and merge that somehow to the final list.



You could also raise the tree size setting temporarily to do that initial
calculation..



Kind regards,

Geert



*Van:* general-boun...@developer.marklogic.com [mailto:
general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
*Verzonden:* maandag 26 maart 2012 14:29


*Aan:* MarkLogic Developer Discussion
*Onderwerp:* Re: [MarkLogic Dev General] Fwd: [1.0-ml]
XDMP-EXPNTREECACHEFULL



Thanks Geert,



Is there any alternate solution to find the unique elements within a
database?



Warm Regards,

Vishnu





On Mon, Mar 26, 2012 at 5:55 PM, Geert Josten  wrote:

Hi Vishnu,



90 mb isn’t much indeed, but MarkLogic is configured to keep a low memory
footprint, even if there are 30 concurrent requests. To make that sure, the
tree size limit (look at the database setting in the admin interface) is
usually pretty low. I have 8Gb and still it is set to no more than 85mb by
default. But you can increase it if you like.



A more streaming approach like my advice attempts to achieve to some extend
helps keeping the footprint

Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

2012-03-26 Thread Szabolcs Illes 
It’s a bit off-topic but you can do that easily with BaseX:

 

Create a DB, load all documents, than:

 

index:element-names("DB_NAME")

 

Szabolcs

 

From: general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Brent Hartwig
Sent: 26 March 2012 14:45
To: sai shanker; MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

 

Curious there aren’t functions like this tapping into the universal index.

 

-Brent

 

From: general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] On Behalf Of sai shanker
Sent: Monday, March 26, 2012 9:23 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

 

 

hi,

you can loop across each document, grab all the child nodes and put them in a 
map.

Thanks and Regards,
Sai.

 

 

From: Ryan Dew 
To: MarkLogic Developer Discussion  
Sent: Monday, March 26, 2012 9:14 AM
Subject: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

 

You could try a recursive function like the following. No guarantee it is 100% 
right, if you have sub elements that have the same names as your root elements. 

 

xquery version "1.0-ml";

 

declare function local:find-unique-qnames($found-qnames as xs:QName*) {

  let $next-qname := cts:search(collection()/*, 

if (exists($found-qnames))

then cts:not-query(cts:element-query($found-qnames,cts:and-query((

else cts:and-query(())

  )[1]/node-name(.)

  return if (exists($next-qname))

  then local:find-unique-qnames(($found-qnames,$next-qname))

  else $found-qnames

};

 

declare function local:find-unique-qnames() {

  for $qn in local:find-unique-qnames(())

  order by string($qn)

  return $qn

};

 

local:find-unique-qnames()

 

On Mon, Mar 26, 2012 at 6:36 AM, Geert Josten  wrote:

Hi Vishnu,

 

It would help if you could explain why you need that list. But in general the 
best option would be to pre-calculate the list I guess. You can save it as a 
server-field (xdmp:set-server-field), to keep the list in memory on each host. 
But you would need an algorithm to initialize it, and each doc commit would 
have to check and update that list. The latter can be done with a post-commit 
trigger. The first can be done best by the strategy I already mentioned: divide 
all docs in chunks of 100 to 1000 docs, calculate distinct names of each chunk, 
and merge that somehow to the final list.

 

You could also raise the tree size setting temporarily to do that initial 
calculation..

 

Kind regards,

Geert

 

Van: general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] Namens VISH RAJPUT
Verzonden: maandag 26 maart 2012 14:29


Aan: MarkLogic Developer Discussion
Onderwerp: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

 

Thanks Geert,

 

Is there any alternate solution to find the unique elements within a database?

 

Warm Regards,

Vishnu

 

 

On Mon, Mar 26, 2012 at 5:55 PM, Geert Josten  wrote:

Hi Vishnu,

 

90 mb isn’t much indeed, but MarkLogic is configured to keep a low memory 
footprint, even if there are 30 concurrent requests. To make that sure, the 
tree size limit (look at the database setting in the admin interface) is 
usually pretty low. I have 8Gb and still it is set to no more than 85mb by 
default. But you can increase it if you like.

 

A more streaming approach like my advice attempts to achieve to some extend 
helps keeping the footprint low, and keep MarkLogic fast.

 

Kind regards,

Geert

 

Van: general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] Namens VISH RAJPUT
Verzonden: maandag 26 maart 2012 14:17
Aan: MarkLogic Developer Discussion
Onderwerp: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

 

Thanks Geert,

 

But still it shows XDMP-EXPNTREECACHEFULL: 
distinct-values(collection("ContentAnalysis")//*/local-name()) -- Expanded tree 
cache full on host the database overall size is only 90MB i don't think it 
is so huge data for marklogic

 

 

Regards,

Vishnu

 

On Mon, Mar 26, 2012 at 1:25 PM, Geert Josten  wrote:

Hi Vishnu,

 

Your FLWOR expression won’t return distinct names, since you are applying the 
function to each individual name. You should write:

 

distinct-values(

for $a in //*

return $a

)

 

Or better:

 

distinct-values(collection()//*/local-name())

 

But this still might not perform well, or still max out on list or tree caches. 
This approach is creating a complete list of all element names first, and 
starts applying distinct-values only thereafter. You might consider taking 
multiple steps, like per doc first, and then clustering per 100 files, and only 
then all clusters. You could also just take 100 random samples, and use that. 
That doesn’t guarantee a 100% complete list, but it remains performant even if 
you

Re: [MarkLogic Dev General] Suggestion Source case sensitivity

2012-03-26 Thread matt.broekhuis 
Yes, I will try this. I was hoping there was just a flag I was missing on the 
options. It seems like something that would be a fairly common use case for 
something that's handling user input like this.

From: Clark Richey [mailto:clark.ric...@marklogic.com]
Sent: Friday, March 23, 2012 9:42 PM
To: MarkLogic Developer Discussion; Broekhuis, Matt
Subject: Re: [MarkLogic Dev General] Suggestion Source case sensitivity

Could you just use some custom JavaScript to parse the search box and when 
there is a search constraint (something followed by a colon), capitalize the 
first letter before sending it to the server for the search:suggest results?
Clark D. Richey, Jr.
Principal  Technologist / SE Manager DoD
MarkLogic Corporation
clark.ric...@marklogic.com
Cell:  +1 240 252 7507
www.marklogic.com
This e-mail and any accompanying attachments are confidential. The information 
is intended solely for the use of the individual to whom it is addressed. Any 
review, disclosure, copying, distribution, or use of this e-mail communication 
by others is strictly prohibited. If you are not the intended recipient, please 
notify us immediately by returning this message to the sender and delete all 
copies. Thank you for your cooperation.

From: 
"matt.broekh...@thomsonreuters.com" 
mailto:matt.broekh...@thomsonreuters.com>>
Reply-To: MarkLogic Developer Discussion 
mailto:general@developer.marklogic.com>>
Date: Fri, 23 Mar 2012 12:50:17 -0700
To: "general@developer.marklogic.com" 
mailto:general@developer.marklogic.com>>
Subject: [MarkLogic Dev General] Suggestion Source case sensitivity

I'm using the search:suggest() api call. I have a large amount of constraints, 
and their affiliated suggestion sources define.

The requirements are such that the constraints should be upper case:





case-insensitive









However, the user will type in

format

with a lower case, and the expectation is that Format: should be suggested. How 
do I make the suggestion source's case insensitive?

___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

2012-03-26 Thread Brent Hartwig 
Curious there aren’t functions like this tapping into the universal index.

-Brent

From: general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] On Behalf Of sai shanker
Sent: Monday, March 26, 2012 9:23 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL


hi,
you can loop across each document, grab all the child nodes and put them in a 
map.
Thanks and Regards,
Sai.


From: Ryan Dew mailto:ryan.j@gmail.com>>
To: MarkLogic Developer Discussion 
mailto:general@developer.marklogic.com>>
Sent: Monday, March 26, 2012 9:14 AM
Subject: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

You could try a recursive function like the following. No guarantee it is 100% 
right, if you have sub elements that have the same names as your root elements.

xquery version "1.0-ml";

declare function local:find-unique-qnames($found-qnames as xs:QName*) {
  let $next-qname := cts:search(collection()/*,
if (exists($found-qnames))
then cts:not-query(cts:element-query($found-qnames,cts:and-query((
else cts:and-query(())
  )[1]/node-name(.)
  return if (exists($next-qname))
  then local:find-unique-qnames(($found-qnames,$next-qname))
  else $found-qnames
};

declare function local:find-unique-qnames() {
  for $qn in local:find-unique-qnames(())
  order by string($qn)
  return $qn
};

local:find-unique-qnames()

On Mon, Mar 26, 2012 at 6:36 AM, Geert Josten 
mailto:geert.jos...@dayon.nl>> wrote:

Hi Vishnu,

It would help if you could explain why you need that list. But in general the 
best option would be to pre-calculate the list I guess. You can save it as a 
server-field (xdmp:set-server-field), to keep the list in memory on each host. 
But you would need an algorithm to initialize it, and each doc commit would 
have to check and update that list. The latter can be done with a post-commit 
trigger. The first can be done best by the strategy I already mentioned: divide 
all docs in chunks of 100 to 1000 docs, calculate distinct names of each chunk, 
and merge that somehow to the final list.

You could also raise the tree size setting temporarily to do that initial 
calculation..

Kind regards,
Geert

Van: 
general-boun...@developer.marklogic.com
 
[mailto:general-boun...@developer.marklogic.com]
 Namens VISH RAJPUT
Verzonden: maandag 26 maart 2012 14:29

Aan: MarkLogic Developer Discussion
Onderwerp: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

Thanks Geert,

Is there any alternate solution to find the unique elements within a database?

Warm Regards,
Vishnu


On Mon, Mar 26, 2012 at 5:55 PM, Geert Josten 
mailto:geert.jos...@dayon.nl>> wrote:
Hi Vishnu,

90 mb isn’t much indeed, but MarkLogic is configured to keep a low memory 
footprint, even if there are 30 concurrent requests. To make that sure, the 
tree size limit (look at the database setting in the admin interface) is 
usually pretty low. I have 8Gb and still it is set to no more than 85mb by 
default. But you can increase it if you like.

A more streaming approach like my advice attempts to achieve to some extend 
helps keeping the footprint low, and keep MarkLogic fast.

Kind regards,
Geert

Van: 
general-boun...@developer.marklogic.com
 
[mailto:general-boun...@developer.marklogic.com]
 Namens VISH RAJPUT
Verzonden: maandag 26 maart 2012 14:17
Aan: MarkLogic Developer Discussion
Onderwerp: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

Thanks Geert,

But still it shows XDMP-EXPNTREECACHEFULL: 
distinct-values(collection("ContentAnalysis")//*/local-name()) -- Expanded tree 
cache full on host the database overall size is only 90MB i don't think it 
is so huge data for marklogic


Regards,
Vishnu

On Mon, Mar 26, 2012 at 1:25 PM, Geert Josten 
mailto:geert.jos...@dayon.nl>> wrote:
Hi Vishnu,

Your FLWOR expression won’t return distinct names, since you are applying the 
function to each individual name. You should write:

distinct-values(
for $a in //*
return $a
)

Or better:

distinct-values(collection()//*/local-name())

But this still might not perform well, or still max out on list or tree caches. 
This approach is creating a complete list of all element names first, and 
starts applying distinct-values only thereafter. You might consider taking 
multiple steps, like per doc first, and then clustering per 100 files, and only 
then all clusters. You could also just take 100 random samples, and use that. 
That doesn’t guarantee a 100% complete list, but it remains performant even if 
your database grows 10 or 100 fold.

Kind regards,
Geert

Van: 
general-boun...@developer.marklogic.com
 
[mailto:general-boun...@developer.marklogic.com

Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

2012-03-26 Thread sai shanker 


hi,
you can loop across each document, grab all the child nodes and put them in a 
map.
Thanks and Regards,
Sai.
 
 
From: Ryan Dew 
To: MarkLogic Developer Discussion  
Sent: Monday, March 26, 2012 9:14 AM
Subject: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL


You could try a recursive function like the following. No guarantee it is 100% 
right, if you have sub elements that have the same names as your root elements. 

xquery version "1.0-ml";

declare function local:find-unique-qnames($found-qnames as xs:QName*) {
  let $next-qname := cts:search(collection()/*, 
    if (exists($found-qnames))
    then cts:not-query(cts:element-query($found-qnames,cts:and-query((
    else cts:and-query(())
  )[1]/node-name(.)
  return if (exists($next-qname))
          then local:find-unique-qnames(($found-qnames,$next-qname))
          else $found-qnames
};

declare function local:find-unique-qnames() {
  for $qn in local:find-unique-qnames(())
  order by string($qn)
  return $qn
};

local:find-unique-qnames()

On Mon, Mar 26, 2012 at 6:36 AM, Geert Josten  wrote:

Hi Vishnu,
> 
>It would help if you could explain why you need that list. But in general the 
>best option would be to pre-calculate the list I guess. You can save it as a 
>server-field (xdmp:set-server-field), to keep the list in memory on each host. 
>But you would need an algorithm to initialize it, and each doc commit would 
>have to check and update that list. The latter can be done with a post-commit 
>trigger. The first can be done best by the strategy I already mentioned: 
>divide all docs in chunks of 100 to 1000 docs, calculate distinct names of 
>each chunk, and merge that somehow to the final list.
> 
>You could also raise the tree size setting temporarily to do that initial 
>calculation..
> 
>Kind regards,
>Geert
> 
>Van:general-boun...@developer.marklogic.com 
>[mailto:general-boun...@developer.marklogic.com] Namens VISH RAJPUT
>Verzonden: maandag 26 maart 2012 14:29
>
>Aan: MarkLogic Developer Discussion
>Onderwerp: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL
> 
>Thanks Geert,
> 
>Is there any alternate solution to find the unique elements within a database?
> 
>Warm Regards,
>Vishnu
> 
> 
>On Mon, Mar 26, 2012 at 5:55 PM, Geert Josten  wrote:
>Hi Vishnu,
> 
>90 mb isn’t much indeed, but MarkLogic is configured to keep a low memory 
>footprint, even if there are 30 concurrent requests. To make that sure, the 
>tree size limit (look at the database setting in the admin interface) is 
>usually pretty low. I have 8Gb and still it is set to no more than 85mb by 
>default. But you can increase it if you like.
> 
>A more streaming approach like my advice attempts to achieve to some extend 
>helps keeping the footprint low, and keep MarkLogic fast.
> 
>Kind regards,
>Geert
> 
>Van:general-boun...@developer.marklogic.com 
>[mailto:general-boun...@developer.marklogic.com] Namens VISH RAJPUT
>Verzonden: maandag 26 maart 2012 14:17
>Aan: MarkLogic Developer Discussion
>Onderwerp: Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL
> 
>Thanks Geert,
> 
>But still it 
>shows XDMP-EXPNTREECACHEFULL: distinct-values(collection("ContentAnalysis")//*/local-name()) --
> Expanded tree cache full on host the database overall size is only 90MB i 
>don't think it is so huge data for marklogic
> 
> 
>Regards,
>Vishnu
> 
>On Mon, Mar 26, 2012 at 1:25 PM, Geert Josten  wrote:
>Hi Vishnu,
> 
>Your FLWOR expression won’t return distinct names, since you are applying the 
>function to each individual name. You should write:
> 
>distinct-values(
>    for $a in //*
>    return $a
>)
> 
>Or better:
> 
>distinct-values(collection()//*/local-name())
> 
>But this still might not perform well, or still max out on list or tree 
>caches. This approach is creating a complete list of all element names first, 
>and starts applying distinct-values only thereafter. You might consider taking 
>multiple steps, like per doc first, and then clustering per 100 files, and 
>only then all clusters. You could also just take 100 random samples, and use 
>that. That doesn’t guarantee a 100% complete list, but it remains performant 
>even if your database grows 10 or 100 fold.
> 
>Kind regards,
>Geert
> 
>Van:general-boun...@developer.marklogic.com 
>[mailto:general-boun...@developer.marklogic.com] Namens VISH RAJPUT
>Verzonden: maandag 26 maart 2012 8:29
>Aan: general@developer.marklogic.com
>Onderwerp: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL
> 
>The size of the all files is 90 MB approx.
>-- Forwarded message --
>From: VISH RAJPUT 
>Date: Mon, Mar 26, 2012 at 11:56 AM
>Subject: [1.0-ml] XDMP-EXPNTREECACHEFULL
>To: general@developer.marklogic.com
>
>
>Hi,
> 
>I have 2000 files in Marklogic database within a single forest and i want to 
>find out the unique element name from this database for the whole 2000 files. 
>For this i wrote the below query:-
> 
>for $a in //*
>return distinct-values

Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

2012-03-26 Thread Ryan Dew 
You could try a recursive function like the following. No guarantee it is
100% right, if you have sub elements that have the same names as your root
elements.

xquery version "1.0-ml";

declare function local:find-unique-qnames($found-qnames as xs:QName*) {
  let $next-qname := cts:search(collection()/*,
if (exists($found-qnames))
then cts:not-query(cts:element-query($found-qnames,cts:and-query((
else cts:and-query(())
  )[1]/node-name(.)
  return if (exists($next-qname))
  then local:find-unique-qnames(($found-qnames,$next-qname))
  else $found-qnames
};

declare function local:find-unique-qnames() {
  for $qn in local:find-unique-qnames(())
  order by string($qn)
  return $qn
};

local:find-unique-qnames()

On Mon, Mar 26, 2012 at 6:36 AM, Geert Josten  wrote:

> Hi Vishnu,
>
>
>
> It would help if you could explain why you need that list. But in general
> the best option would be to pre-calculate the list I guess. You can save it
> as a server-field (xdmp:set-server-field), to keep the list in memory on
> each host. But you would need an algorithm to initialize it, and each doc
> commit would have to check and update that list. The latter can be done
> with a post-commit trigger. The first can be done best by the strategy I
> already mentioned: divide all docs in chunks of 100 to 1000 docs, calculate
> distinct names of each chunk, and merge that somehow to the final list.
>
>
>
> You could also raise the tree size setting temporarily to do that initial
> calculation..
>
>
>
> Kind regards,
>
> Geert
>
>
>
> *Van:* general-boun...@developer.marklogic.com [mailto:
> general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
> *Verzonden:* maandag 26 maart 2012 14:29
>
> *Aan:* MarkLogic Developer Discussion
> *Onderwerp:* Re: [MarkLogic Dev General] Fwd: [1.0-ml]
> XDMP-EXPNTREECACHEFULL
>
>
>
> Thanks Geert,
>
>
>
> Is there any alternate solution to find the unique elements within a
> database?
>
>
>
> Warm Regards,
>
> Vishnu
>
>
>
>
>
> On Mon, Mar 26, 2012 at 5:55 PM, Geert Josten 
> wrote:
>
> Hi Vishnu,
>
>
>
> 90 mb isn’t much indeed, but MarkLogic is configured to keep a low memory
> footprint, even if there are 30 concurrent requests. To make that sure, the
> tree size limit (look at the database setting in the admin interface) is
> usually pretty low. I have 8Gb and still it is set to no more than 85mb by
> default. But you can increase it if you like.
>
>
>
> A more streaming approach like my advice attempts to achieve to some
> extend helps keeping the footprint low, and keep MarkLogic fast.
>
>
>
> Kind regards,
>
> Geert
>
>
>
> *Van:* general-boun...@developer.marklogic.com [mailto:
> general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
> *Verzonden:* maandag 26 maart 2012 14:17
> *Aan:* MarkLogic Developer Discussion
> *Onderwerp:* Re: [MarkLogic Dev General] Fwd: [1.0-ml]
> XDMP-EXPNTREECACHEFULL
>
>
>
> Thanks Geert,
>
>
>
> But still it shows *XDMP-EXPNTREECACHEFULL: 
> distinct-values(collection("ContentAnalysis")//*/local-name()) --
> Expanded tree cache full on host *the database overall size is only
> 90MB i don't think it is so huge data for marklogic
>
>
>
>
>
> Regards,
>
> Vishnu
>
>
>
> On Mon, Mar 26, 2012 at 1:25 PM, Geert Josten 
> wrote:
>
> Hi Vishnu,
>
>
>
> Your FLWOR expression won’t return distinct names, since you are applying
> the function to each individual name. You should write:
>
>
>
> distinct-values(
>
> for $a in //*
>
> return $a
>
> )
>
>
>
> Or better:
>
>
>
> distinct-values(collection()//*/local-name())
>
>
>
> But this still might not perform well, or still max out on list or tree
> caches. This approach is creating a complete list of all element names
> first, and starts applying distinct-values only thereafter. You might
> consider taking multiple steps, like per doc first, and then clustering per
> 100 files, and only then all clusters. You could also just take 100 random
> samples, and use that. That doesn’t guarantee a 100% complete list, but it
> remains performant even if your database grows 10 or 100 fold.
>
>
>
> Kind regards,
>
> Geert
>
>
>
> *Van:* general-boun...@developer.marklogic.com [mailto:
> general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
> *Verzonden:* maandag 26 maart 2012 8:29
> *Aan:* general@developer.marklogic.com
> *Onderwerp:* [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL
>
>
>
> The size of the all files is 90 MB approx.
>
> -- Forwarded message --
> From: *VISH RAJPUT* 
> Date: Mon, Mar 26, 2012 at 11:56 AM
> Subject: [1.0-ml] XDMP-EXPNTREECACHEFULL
> To: general@developer.marklogic.com
>
>
> Hi,
>
>
>
> I have 2000 files in Marklogic database within a single forest and i want
> to find out the unique element name from this database for the whole 2000
> files. For this i wrote the below query:-
>
>
>
> for $a in //*
>
> return distinct-values($a/local-name()))
>
>
>
> but by this i got an error "*[1.0-ml] XDMP-EXPNTREEC

Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

2012-03-26 Thread Geert Josten 
Hi Vishnu,



It would help if you could explain why you need that list. But in general
the best option would be to pre-calculate the list I guess. You can save it
as a server-field (xdmp:set-server-field), to keep the list in memory on
each host. But you would need an algorithm to initialize it, and each doc
commit would have to check and update that list. The latter can be done
with a post-commit trigger. The first can be done best by the strategy I
already mentioned: divide all docs in chunks of 100 to 1000 docs, calculate
distinct names of each chunk, and merge that somehow to the final list.



You could also raise the tree size setting temporarily to do that initial
calculation..



Kind regards,

Geert



*Van:* general-boun...@developer.marklogic.com [mailto:
general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
*Verzonden:* maandag 26 maart 2012 14:29
*Aan:* MarkLogic Developer Discussion
*Onderwerp:* Re: [MarkLogic Dev General] Fwd: [1.0-ml]
XDMP-EXPNTREECACHEFULL



Thanks Geert,



Is there any alternate solution to find the unique elements within a
database?



Warm Regards,

Vishnu





On Mon, Mar 26, 2012 at 5:55 PM, Geert Josten  wrote:

Hi Vishnu,



90 mb isn’t much indeed, but MarkLogic is configured to keep a low memory
footprint, even if there are 30 concurrent requests. To make that sure, the
tree size limit (look at the database setting in the admin interface) is
usually pretty low. I have 8Gb and still it is set to no more than 85mb by
default. But you can increase it if you like.



A more streaming approach like my advice attempts to achieve to some extend
helps keeping the footprint low, and keep MarkLogic fast.



Kind regards,

Geert



*Van:* general-boun...@developer.marklogic.com [mailto:
general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
*Verzonden:* maandag 26 maart 2012 14:17
*Aan:* MarkLogic Developer Discussion
*Onderwerp:* Re: [MarkLogic Dev General] Fwd: [1.0-ml]
XDMP-EXPNTREECACHEFULL



Thanks Geert,



But still it shows *XDMP-EXPNTREECACHEFULL:
distinct-values(collection("ContentAnalysis")//*/local-name()) --
Expanded tree cache full on host *the database overall size is only
90MB i don't think it is so huge data for marklogic





Regards,

Vishnu



On Mon, Mar 26, 2012 at 1:25 PM, Geert Josten  wrote:

Hi Vishnu,



Your FLWOR expression won’t return distinct names, since you are applying
the function to each individual name. You should write:



distinct-values(

for $a in //*

return $a

)



Or better:



distinct-values(collection()//*/local-name())



But this still might not perform well, or still max out on list or tree
caches. This approach is creating a complete list of all element names
first, and starts applying distinct-values only thereafter. You might
consider taking multiple steps, like per doc first, and then clustering per
100 files, and only then all clusters. You could also just take 100 random
samples, and use that. That doesn’t guarantee a 100% complete list, but it
remains performant even if your database grows 10 or 100 fold.



Kind regards,

Geert



*Van:* general-boun...@developer.marklogic.com [mailto:
general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
*Verzonden:* maandag 26 maart 2012 8:29
*Aan:* general@developer.marklogic.com
*Onderwerp:* [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL



The size of the all files is 90 MB approx.

-- Forwarded message --
From: *VISH RAJPUT* 
Date: Mon, Mar 26, 2012 at 11:56 AM
Subject: [1.0-ml] XDMP-EXPNTREECACHEFULL
To: general@developer.marklogic.com


Hi,



I have 2000 files in Marklogic database within a single forest and i want
to find out the unique element name from this database for the whole 2000
files. For this i wrote the below query:-



for $a in //*

return distinct-values($a/local-name()))



but by this i got an error "*[1.0-ml] XDMP-EXPNTREECACHEFULL" * what should
i do?





Regards,

Vishnu Singh




___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general




___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

2012-03-26 Thread VISH RAJPUT 
Thanks Geert,

Is there any alternate solution to find the unique elements within a
database?

Warm Regards,
Vishnu



On Mon, Mar 26, 2012 at 5:55 PM, Geert Josten  wrote:

> Hi Vishnu,
>
>
>
> 90 mb isn’t much indeed, but MarkLogic is configured to keep a low memory
> footprint, even if there are 30 concurrent requests. To make that sure, the
> tree size limit (look at the database setting in the admin interface) is
> usually pretty low. I have 8Gb and still it is set to no more than 85mb by
> default. But you can increase it if you like.
>
>
>
> A more streaming approach like my advice attempts to achieve to some
> extend helps keeping the footprint low, and keep MarkLogic fast.
>
>
>
> Kind regards,
>
> Geert
>
>
>
> *Van:* general-boun...@developer.marklogic.com [mailto:
> general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
> *Verzonden:* maandag 26 maart 2012 14:17
> *Aan:* MarkLogic Developer Discussion
> *Onderwerp:* Re: [MarkLogic Dev General] Fwd: [1.0-ml]
> XDMP-EXPNTREECACHEFULL
>
>
>
> Thanks Geert,
>
>
>
> But still it shows *XDMP-EXPNTREECACHEFULL: 
> distinct-values(collection("ContentAnalysis")//*/local-name()) --
> Expanded tree cache full on host *the database overall size is only
> 90MB i don't think it is so huge data for marklogic
>
>
>
>
>
> Regards,
>
> Vishnu
>
>
>
> On Mon, Mar 26, 2012 at 1:25 PM, Geert Josten 
> wrote:
>
> Hi Vishnu,
>
>
>
> Your FLWOR expression won’t return distinct names, since you are applying
> the function to each individual name. You should write:
>
>
>
> distinct-values(
>
> for $a in //*
>
> return $a
>
> )
>
>
>
> Or better:
>
>
>
> distinct-values(collection()//*/local-name())
>
>
>
> But this still might not perform well, or still max out on list or tree
> caches. This approach is creating a complete list of all element names
> first, and starts applying distinct-values only thereafter. You might
> consider taking multiple steps, like per doc first, and then clustering per
> 100 files, and only then all clusters. You could also just take 100 random
> samples, and use that. That doesn’t guarantee a 100% complete list, but it
> remains performant even if your database grows 10 or 100 fold.
>
>
>
> Kind regards,
>
> Geert
>
>
>
> *Van:* general-boun...@developer.marklogic.com [mailto:
> general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
> *Verzonden:* maandag 26 maart 2012 8:29
> *Aan:* general@developer.marklogic.com
> *Onderwerp:* [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL
>
>
>
> The size of the all files is 90 MB approx.
>
> -- Forwarded message --
> From: *VISH RAJPUT* 
> Date: Mon, Mar 26, 2012 at 11:56 AM
> Subject: [1.0-ml] XDMP-EXPNTREECACHEFULL
> To: general@developer.marklogic.com
>
>
> Hi,
>
>
>
> I have 2000 files in Marklogic database within a single forest and i want
> to find out the unique element name from this database for the whole 2000
> files. For this i wrote the below query:-
>
>
>
> for $a in //*
>
> return distinct-values($a/local-name()))
>
>
>
> but by this i got an error "*[1.0-ml] XDMP-EXPNTREECACHEFULL" * what
> should i do?
>
>
>
>
>
> Regards,
>
> Vishnu Singh
>
>
>
>
> ___
> General mailing list
> General@developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general
>
>
>
> ___
> General mailing list
> General@developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general
>
>
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

2012-03-26 Thread Geert Josten 
Hi Vishnu,



90 mb isn’t much indeed, but MarkLogic is configured to keep a low memory
footprint, even if there are 30 concurrent requests. To make that sure, the
tree size limit (look at the database setting in the admin interface) is
usually pretty low. I have 8Gb and still it is set to no more than 85mb by
default. But you can increase it if you like.



A more streaming approach like my advice attempts to achieve to some extend
helps keeping the footprint low, and keep MarkLogic fast.



Kind regards,

Geert



*Van:* general-boun...@developer.marklogic.com [mailto:
general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
*Verzonden:* maandag 26 maart 2012 14:17
*Aan:* MarkLogic Developer Discussion
*Onderwerp:* Re: [MarkLogic Dev General] Fwd: [1.0-ml]
XDMP-EXPNTREECACHEFULL



Thanks Geert,



But still it shows *XDMP-EXPNTREECACHEFULL:
distinct-values(collection("ContentAnalysis")//*/local-name()) --
Expanded tree cache full on host *the database overall size is only
90MB i don't think it is so huge data for marklogic





Regards,

Vishnu



On Mon, Mar 26, 2012 at 1:25 PM, Geert Josten  wrote:

Hi Vishnu,



Your FLWOR expression won’t return distinct names, since you are applying
the function to each individual name. You should write:



distinct-values(

for $a in //*

return $a

)



Or better:



distinct-values(collection()//*/local-name())



But this still might not perform well, or still max out on list or tree
caches. This approach is creating a complete list of all element names
first, and starts applying distinct-values only thereafter. You might
consider taking multiple steps, like per doc first, and then clustering per
100 files, and only then all clusters. You could also just take 100 random
samples, and use that. That doesn’t guarantee a 100% complete list, but it
remains performant even if your database grows 10 or 100 fold.



Kind regards,

Geert



*Van:* general-boun...@developer.marklogic.com [mailto:
general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
*Verzonden:* maandag 26 maart 2012 8:29
*Aan:* general@developer.marklogic.com
*Onderwerp:* [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL



The size of the all files is 90 MB approx.

-- Forwarded message --
From: *VISH RAJPUT* 
Date: Mon, Mar 26, 2012 at 11:56 AM
Subject: [1.0-ml] XDMP-EXPNTREECACHEFULL
To: general@developer.marklogic.com


Hi,



I have 2000 files in Marklogic database within a single forest and i want
to find out the unique element name from this database for the whole 2000
files. For this i wrote the below query:-



for $a in //*

return distinct-values($a/local-name()))



but by this i got an error "*[1.0-ml] XDMP-EXPNTREECACHEFULL" * what should
i do?





Regards,

Vishnu Singh




___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

2012-03-26 Thread VISH RAJPUT 
Thanks Geert,

But still it shows *XDMP-EXPNTREECACHEFULL:
distinct-values(collection("ContentAnalysis")//*/local-name()) --
Expanded tree cache full on host* the database overall size is only
90MB i don't think it is so huge data for marklogic


Regards,
Vishnu


On Mon, Mar 26, 2012 at 1:25 PM, Geert Josten  wrote:

> Hi Vishnu,
>
>
>
> Your FLWOR expression won’t return distinct names, since you are applying
> the function to each individual name. You should write:
>
>
>
> distinct-values(
>
> for $a in //*
>
> return $a
>
> )
>
>
>
> Or better:
>
>
>
> distinct-values(collection()//*/local-name())
>
>
>
> But this still might not perform well, or still max out on list or tree
> caches. This approach is creating a complete list of all element names
> first, and starts applying distinct-values only thereafter. You might
> consider taking multiple steps, like per doc first, and then clustering per
> 100 files, and only then all clusters. You could also just take 100 random
> samples, and use that. That doesn’t guarantee a 100% complete list, but it
> remains performant even if your database grows 10 or 100 fold.
>
>
>
> Kind regards,
>
> Geert
>
>
>
> *Van:* general-boun...@developer.marklogic.com [mailto:
> general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
> *Verzonden:* maandag 26 maart 2012 8:29
> *Aan:* general@developer.marklogic.com
> *Onderwerp:* [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL
>
>
>
> The size of the all files is 90 MB approx.
>
> -- Forwarded message --
> From: *VISH RAJPUT* 
> Date: Mon, Mar 26, 2012 at 11:56 AM
> Subject: [1.0-ml] XDMP-EXPNTREECACHEFULL
> To: general@developer.marklogic.com
>
>
> Hi,
>
>
>
> I have 2000 files in Marklogic database within a single forest and i want
> to find out the unique element name from this database for the whole 2000
> files. For this i wrote the below query:-
>
>
>
> for $a in //*
>
> return distinct-values($a/local-name()))
>
>
>
> but by this i got an error "*[1.0-ml] XDMP-EXPNTREECACHEFULL" * what
> should i do?
>
>
>
>
>
> Regards,
>
> Vishnu Singh
>
>
>
> ___
> General mailing list
> General@developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general
>
>
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL

2012-03-26 Thread Geert Josten 
Hi Vishnu,



Your FLWOR expression won’t return distinct names, since you are applying
the function to each individual name. You should write:



distinct-values(

for $a in //*

return $a

)



Or better:



distinct-values(collection()//*/local-name())



But this still might not perform well, or still max out on list or tree
caches. This approach is creating a complete list of all element names
first, and starts applying distinct-values only thereafter. You might
consider taking multiple steps, like per doc first, and then clustering per
100 files, and only then all clusters. You could also just take 100 random
samples, and use that. That doesn’t guarantee a 100% complete list, but it
remains performant even if your database grows 10 or 100 fold.



Kind regards,

Geert



*Van:* general-boun...@developer.marklogic.com [mailto:
general-boun...@developer.marklogic.com] *Namens *VISH RAJPUT
*Verzonden:* maandag 26 maart 2012 8:29
*Aan:* general@developer.marklogic.com
*Onderwerp:* [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL



The size of the all files is 90 MB approx.

-- Forwarded message --
From: *VISH RAJPUT* 
Date: Mon, Mar 26, 2012 at 11:56 AM
Subject: [1.0-ml] XDMP-EXPNTREECACHEFULL
To: general@developer.marklogic.com


Hi,



I have 2000 files in Marklogic database within a single forest and i want
to find out the unique element name from this database for the whole 2000
files. For this i wrote the below query:-



for $a in //*

return distinct-values($a/local-name()))



but by this i got an error "*[1.0-ml] XDMP-EXPNTREECACHEFULL" * what should
i do?





Regards,

Vishnu Singh
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general