Re: [MarkLogic Dev General] Adding new fields

[ville]

Thank you for the suggestions!

We have a meta and a body element in all documents. The new field must 
index /doc/body[@identifier_id="some_id"], so the element is found from 
all documents, but the include in the field limits with the attribute 
value. It seems that it reindexes all documents in this case, even if 
only fraction of documents have "some_id" (Or none. If I add the index 
beforehand, it still reindexes all content, even though not a single doc 
has @identifier="non-existent-id").


I will fiddle with the path configurations in our dev environment, and 
see if using that helps.


Ville

-- Original Message --
From: "Danny Sokolsky" 
To: "MarkLogic Developer Discussion" 
Sent: 14.10.2014 21:48:03
Subject: Re: [MarkLogic Dev General] Adding new fields

One other thing here:  if you have the reindexer off, you can ask the 
system how many fragments need reindexing using the “preview-reindexer” 
option in xdmp:forest-counts.  For example:




xdmp:forest-counts(xdmp:database-forests(xdmp:database("Documents")), 
(), "preview-reindexer")




This will query the database and calculate the number of fragments 
needing reindexing, returning a report.




-Danny



From: Danny Sokolsky
Sent: Tuesday, October 14, 2014 11:14 AM
To: 'vi...@tilaton.fi'; 'MarkLogic Developer Discussion'
Subject: RE: [MarkLogic Dev General] Adding new fields



You can turn reindexing off during peak times to minimize the impact.  
It will pick up where you left off when you turn it back on.  Another 
thing you can do is to leave reindexing off, but just rewrite the 
documents that you want to (for example, do a document-insert of a 
document with its previous content as what to insert)—that will have 
the effect of reindexing just those documents.




If you use the path to specify your field, you can use any path that 
returns true from cts:valid-index-path:




http://docs.marklogic.com/cts:valid-index-path



I am not really understanding how your field can affect every document 
but you only want it in some of the documents.  Maybe the field is not 
selective enough (the path field might help there)?




Also, there are several bug fixes in 7.0-4 wrt fields, so planning and 
testing an upgrade might be a good idea.




-Danny





From:general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] On Behalf Of 
vi...@tilaton.fi

Sent: Monday, October 13, 2014 11:43 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Adding new fields



Hi,



The indexes do not include the root element. Unfortunately we need to 
index an element that exists in all documents - the only thing that 
differs is the attribute value. (The field index settings are tweaked 
for specific purposes in each case, result from having different full 
text search requirements for specific document sets.)




As this is built on top of another product, we need to have the element 
named like it is in there, and the element is found in all documents. 
When I look at the database status right after adding one such field, I 
can see that the forests are all reindexing totalling millions of docs 
to go. With new tiered hardware this is completed in order of hours, 
sometimes takes over a day though, and with old hardware it took in 
order of weeks to add one. Our monitoring also reveals that it really 
spikes the usable disk bandwidth, so it is definitely working a lot. 
(My guess is that it selects all the docs with the element, but is not 
intelligent enough to limit using the attribute value too.)




Indexes that include only elements that can be found from a fraction of 
documents are not a problem. Is there some indexing option that I can 
turn on so that ML can index only the docs that have a specific 
attribute value in the given element? Now it seems only capable of 
querying the docs that have the element




This may also be a design issue, but unfortunately I'm unable to do any 
big changes to the way we do things in the codebase I've inherited.




We're running 7.0-2.3 btw, if that matters.



Ville



-- Original Message --

From: "Danny Sokolsky" 

To: "MarkLogic Developer Discussion" 

Sent: 14.10.2014 0:41:51

Subject: Re: [MarkLogic Dev General] Adding new fields




Hi Ville,



I don’t know of a way to tell MarkLogic to trust you in this case, and 
you should not need it to.  If you do not have any content to reindex, 
and if reindexing is enabled, it should not rewrite all of the 
content.  It will query all of the content to see if it needs 
reindexing, which will not be free but should not be too expensive, 
but I would not expect a full reindex to happen.  In that case you 
should see some messages in the log about reindexing that database and 
a little later another message saying you reindexes 0 fragments (in 
fact, you will see these messages each time the config files change).




You mention your fields are doing includes.  I would recommend using 
paths for 

Re: [MarkLogic Dev General] xdmp:output serialization options not working?

[David Sewell]

Thanks, David. Whipping up a schema to associate with the document in question 
turned out to be the easiest solution, and it worked perfectly.

David S.

On Tue, 14 Oct 2014, David Lee wrote:

> There isnt an 'easy' way that I know of.
>
> The reason your docuements are indented nicely is because your XML editor did 
> so, and the indentation (newlines, spaces etc) are already in the file.
>
> You can create such spaces yourself very carefully ... and by making sure the 
> right XQuery options are in set, such as
>
>
>
> declare boundary-space preserve;
>
>
>
> Otherwise even your best attempts will fail ... but that won't *add 
> indentation* ...
>
>
>
> Example:
>
>
>
> declare boundary-space strip;
>
> let $x :=
>
> 
>
>   
>
> {"a  ccc  xx"}
>
> return xdmp:document-insert("/test.xml",$x)
>
>
>
>
>
>  Now see what the doc looks like with some xml agnostic code ...
>
> This will fool most processors even QC:
>
>
>
> xdmp:quote(doc("/test.xml"))
>
>
>
>
>
> 
>
> a  ccc  xx
>
>
>
> Spaces gone ... boo hoo
>
> now add
>
> declare boundary-space strip;
>
>
>
> and retest
>
> 
>
> 
>
>   
>
> a  ccc  xx
>
>
> Better !
>
>
>
> But still you had to put the spaces in the first place.
>
>
>
> To actually get XML "pretty printed" automatically ... you can do a 
> round-trip through xquery or xslt to take advantage of the output indenting 
> formatter (UG !)
>
>
>
> There is one other way I know of -- If you have a schema in-scope then the 
> document is indentedon insert
>
>
>
> Example: I happen to be working on a twitter dump where I have a schema ... 
> The indentation isnt perfect but without a schema
>
> its pretty much impossible to tell what kinds of whitespace  are allowed to 
> be inserted  (that's why ML doesn't do it ...
>
> it can break your data !)
>
>
>
> With a schema in scope ... ML indents ...
>
> you can tell with xdmp:quote() or by getting the file out in text mode
>
>
>
>
>
> let $x :=
>
> http://www.xmlsh.org/schemas/twitter";> id="298616637173686273"> description=""lang="en"name="Martens VW"
>
>screen-name="martens_vw">
>
> 
>
>
>
> return ( xdmp:document-insert("/test.xml",$x))
>
>
>
>
>
> This proces (with my schema in place)
>
>
>
> 
>
> http://www.xmlsh.org/schemas/twitter";>
>
>  
>
> name="Martens VW" screen-name="martens_vw">
>
>  
>
>
>
>  
>
> 
>
>
>
> David Lee
>
> Lead Engineer
>
> MarkLogic Corporation
>
> d...@marklogic.com
>
> Phone: +1 812-482-5224
>
> Cell:  +1 812-630-7622
>
> www.marklogic.com
>
>
>
> -Original Message-
> From: general-boun...@developer.marklogic.com 
> [mailto:general-boun...@developer.marklogic.com] On Behalf Of David Sewell
> Sent: Tuesday, October 14, 2014 3:57 PM
> To: MarkLogic Developer Discussion
> Subject: Re: [MarkLogic Dev General] xdmp:output serialization options not 
> working?
>
>
>
> Let me restate the problem.
>
>
>
> If I use an editor to manually create this XML document and save it to 
> MarkLogic via WebDav or other means:
>
>
>
> 
>
> 
>
>   line 1
>
>   line2
>
> 
>
>
>
> then when I subsequently open that document from the database via WebDav or a
>
> doc() call, it arrives with its indentation preserved. I want to know how to 
> generate a new file via XQuery such that it will have indentation when 
> subsquently opened via WebDav or a doc() call.
>
>
>
> (It's not mission-critical, but I'm auto-generating a 50MB data file that I 
> would like to be able to open in oXygen via WebDav with its elements already 
> indented, since thing go much faster that way.)
>
>
>
> DAvid
>
>
>
> On Tue, 14 Oct 2014, Mary Holstege wrote:
>
>
>
>> On Tue, 14 Oct 2014 12:28:52 -0700, David Sewell
>
>> mailto:dsew...@virginia.edu>>
>
>> wrote:
>
>>
>
>>> Given this code:
>
>>>
>
>>> xquery version "1.0-ml";
>
>>> declare option xdmp:output "indent-untyped=yes"; declare option
>
>>> xdmp:output "omit-xml-declaration=yes"; xdmp:document-insert(
>
>>>"/test.xml",
>
>>>
>
>>>  line 1
>
>>>  line2
>
>>>
>
>>> ) ;
>
>>>
>
>>> doc("/test.xml")
>
>>>
>
>>> The output I'm getting is
>
>>>
>
>>>  line
>
>>> 1line2
>
>>>
>
>>> which is disobeying both serialization optinons I specified. ML
>
>>> version 7.0-4.
>
>>> Are other people seeing this? Am I missing something?
>
>>>
>
>>> David
>
>>>
>
>>
>
>>
>
>> Two things:
>
>> (1) If you are running this in QConsole all bets are off, because the
>
>> serialization options apply in the query that does the serialization
>
>> and everything in QConsole is run in a separate eval.
>
>>
>
>> (2) The serialization options are before the semi-colon, that is, they
>
>> only apply to the xdmp:document-insert part of this file. If you want
>
>> them to apply to the serialization of doc("/test.xml") you need to put
>
>> them into that query.
>
>>
>
>> //Mary
>
>> ___
>
>> General mailing list
>
>> General@developer.marklogic.com
>
>> http://devel

Re: [MarkLogic Dev General] xdmp:output serialization options not working?

[David Ennis]

HI.

To clarify:  MarkLogic does not preserve your indentention, nor does it
guarantee that the document is the exact same upon return.  Yes, the same
content, but normalized as in the return document is actually assembled for
you.  Your document is ripped to bits and stored internally ina  different
format.

As an example(query console):
Insert this:
​​

 line 1
line2


The query it:

line 1
line2


So, nice and pretty - but not actually 'preserved' from my input (notice
the indentation is more consistent and also note that the use of single
quotes was changed to double quotes).


The pretty return that you see is actually done in the query console code.

You can replicate this in your own code by setting the following:
declare option xdmp:output "indent=yes";
declare option xdmp:output "indent-untyped=yes";

More information about this can be found here:


http://developer.marklogic.com/blog/controlling-output-options



Kind Regards,
David Ennis


David Ennis
*Content Engineer*

[image: HintTech]  
Mastering the value of content
creative | technology | content

Delftechpark 37i
2628 XJ Delft
The Netherlands
T: +31 88 268 25 00
M: +31 63 091 72 80

[image: http://www.hinttech.com] 
  


On 14 October 2014 21:56, David Sewell  wrote:

> Let me restate the problem.
>
> If I use an editor to manually create this XML document and save it to
> MarkLogic
> via WebDav or other means:
>
> 
> 
>line 1
>line2
> 
>
> then when I subsequently open that document from the database via WebDav
> or a
> doc() call, it arrives with its indentation preserved. I want to know how
> to
> generate a new file via XQuery such that it will have indentation when
> subsquently opened via WebDav or a doc() call.
>
> (It's not mission-critical, but I'm auto-generating a 50MB data file that I
> would like to be able to open in oXygen via WebDav with its elements
> already
> indented, since thing go much faster that way.)
>
> DAvid
>
> On Tue, 14 Oct 2014, Mary Holstege wrote:
>
> > On Tue, 14 Oct 2014 12:28:52 -0700, David Sewell 
> > wrote:
> >
> >> Given this code:
> >>
> >> xquery version "1.0-ml";
> >> declare option xdmp:output "indent-untyped=yes";
> >> declare option xdmp:output "omit-xml-declaration=yes";
> >> xdmp:document-insert(
> >>"/test.xml",
> >>
> >>  line 1
> >>  line2
> >>
> >> ) ;
> >>
> >> doc("/test.xml")
> >>
> >> The output I'm getting is
> >>
> >> 
> >> line 1line2
> >>
> >> which is disobeying both serialization optinons I specified. ML version
> >> 7.0-4.
> >> Are other people seeing this? Am I missing something?
> >>
> >> David
> >>
> >
> >
> > Two things:
> > (1) If you are running this in QConsole all bets are off, because the
> > serialization
> > options apply in the query that does the serialization and everything in
> > QConsole
> > is run in a separate eval.
> >
> > (2) The serialization options are before the semi-colon, that is, they
> > only apply to
> > the xdmp:document-insert part of this file. If you want them to apply to
> > the
> > serialization of doc("/test.xml") you need to put them into that query.
> >
> > //Mary
> > ___
> > General mailing list
> > General@developer.marklogic.com
> > http://developer.marklogic.com/mailman/listinfo/general
> >
>
> --
> David Sewell, Editorial and Technical Manager
> ROTUNDA, The University of Virginia Press
> PO Box 400314, Charlottesville, VA 22904-4314 USA
> Email: dsew...@virginia.edu   Tel: +1 434 924 9973
> Web: http://rotunda.upress.virginia.edu/
> ___
> General mailing list
> General@developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general
>
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] How to optimize the REST API Bulk Ingestion Performance?

[Justin Makeig]

The bulk API does not spawn tasks in the Server. It allows you to send groups 
of documents together in a single request. (The implementation is actually HTTP 
multipart.) An E-node can, of course, handle multiple simultaneous requests, 
bulk or not.

Why spawn in the Server? Without knowing much more about what the bottlenecks 
actually are, I'd start by spawning threads in C#, similar the way that 
something like mlcp or Corb does. It sounds like you have some headroom in 
MarkLogic so you should be able to throw more work at it from the client. 

Justin


On Oct 14, 2014, at 12:16 PM, Gary Russo  wrote:

> Hello Danny,
>  
> Yes, I’m using 7.0-4.
>  
> >> What are you comparing it to on the Oracle side?
> >> In MarkLogic, the content will be all indexed and searchable.  Is that 
> >> true on the orcl side too
>  
> The Oracle side is doing a basic CLOB insert with no indexing.
>  
> The Oracle server being compared to is a higher capacity system so we 
> expected to see a faster ingestion.
>  
> I didn’t expect the MarkLogic side to be 4 times slower.
>  
> Yes, we tried tweaking the batch size. The 500 batch size had the fastest 
> load times.
>  
> I will investigate further but I believe the bottleneck is on the MarkLogic 
> side.
>  
> I believe the MarkLogic CPU has some room for parallelizing.
>  
> I’ll create a custom REST Extension that will spawn multiple threads for the 
> doc-inserts.
>  
> I assume the REST API bulk ingestion already does this but I can’t say for 
> sure.
>  
> I’ll keep you posted.
>  
> Thanks Danny
>  
> -  Gary R
>  
>  
>  
> From: general-boun...@developer.marklogic.com 
> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Danny Sokolsky
> Sent: Tuesday, October 14, 2014 2:00 PM
> To: MarkLogic Developer Discussion
> Subject: Re: [MarkLogic Dev General] How to optimize the REST API Bulk 
> Ingestion Performance?
>  
> Hi Gary,
>  
> A few thoughts here.  You are using 7.0-4 on this? 
>  
> What are you comparing it to on the Oracle side?  In MarkLogic, the content 
> will be all indexed and searchable.  Is that true on the orcl side too?
>  
> What indexes to you have enabled?  Maybe you do not need them all (or maybe 
> you should put the equivalent indexing on the orcl side)?
>  
> Have you tried tweaking the batch size?  I would try a smaller number, say 50 
> or 100.
>  
> Have you analyzed where you are spending the time?  In the c# code?  In the 
> code loading the doc on MarkLogic?
>  
> Do you have multiple threads loading from your .net program?  If you are not 
> maxing out your cpu on the MarkLogic side, you probably have room for more 
> parallelization.
>  
> -Danny
>  
> From: general-boun...@developer.marklogic.com 
> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Gary Russo
> Sent: Tuesday, October 14, 2014 9:21 AM
> To: general@developer.marklogic.com
> Subject: [MarkLogic Dev General] How to optimize the REST API Bulk Ingestion 
> Performance?
>  
> MarkLogic Bulk ingestion processing is slower than an equivalent Oracle 
> ingestion process.
>  
> The MarkLogic ingestion takes 30 minutes. An Oracle equivalent only takes 7 
> minutes.
>  
> I’m using the REST API to bulk ingest multiple documents as described here. 
> => http://docs.marklogic.com/guide/rest-dev/bulk#id_54649
>  
> Notes:
> · C# code is used to call the MarkLogic Bulk Ingest REST API.
> · Document batch size used is 500.
> · Average doc size is 1 KB.
> · JSON Conversion and Validation logic occurs in the C# code.
>  
>  
> Any thoughts on how to optimize the MarkLogic bulk ingest to make it as fast 
> as Oracle’s 7 minute load time?
>  
>  
> Thanks,
> Gary R
>  
>  
> Gary Russo
> Enterprise NoSQL Developer
> http://garyrusso.wordpress.com
>  
> ___
> General mailing list
> General@developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general



smime.p7s
Description: S/MIME cryptographic signature
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] xdmp:output serialization options not working?

[David Lee]

There isnt an 'easy' way that I know of.

The reason your docuements are indented nicely is because your XML editor did 
so, and the indentation (newlines, spaces etc) are already in the file.

You can create such spaces yourself very carefully ... and by making sure the 
right XQuery options are in set, such as



declare boundary-space preserve;



Otherwise even your best attempts will fail ... but that won't *add 
indentation* ...



Example:



declare boundary-space strip;

let $x :=



   

 {"a  ccc  xx"}

return xdmp:document-insert("/test.xml",$x)





 Now see what the doc looks like with some xml agnostic code ...

This will fool most processors even QC:



xdmp:quote(doc("/test.xml"))







a  ccc  xx



Spaces gone ... boo hoo

now add

declare boundary-space strip;



and retest





   

 a  ccc  xxhttp://www.xmlsh.org/schemas/twitter";>





return ( xdmp:document-insert("/test.xml",$x))





This proces (with my schema in place)





http://www.xmlsh.org/schemas/twitter";>

  



  



  





David Lee

Lead Engineer

MarkLogic Corporation

d...@marklogic.com

Phone: +1 812-482-5224

Cell:  +1 812-630-7622

www.marklogic.com



-Original Message-
From: general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] On Behalf Of David Sewell
Sent: Tuesday, October 14, 2014 3:57 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] xdmp:output serialization options not 
working?



Let me restate the problem.



If I use an editor to manually create this XML document and save it to 
MarkLogic via WebDav or other means:







   line 1

   line2





then when I subsequently open that document from the database via WebDav or a

doc() call, it arrives with its indentation preserved. I want to know how to 
generate a new file via XQuery such that it will have indentation when 
subsquently opened via WebDav or a doc() call.



(It's not mission-critical, but I'm auto-generating a 50MB data file that I 
would like to be able to open in oXygen via WebDav with its elements already 
indented, since thing go much faster that way.)



DAvid



On Tue, 14 Oct 2014, Mary Holstege wrote:



> On Tue, 14 Oct 2014 12:28:52 -0700, David Sewell

> mailto:dsew...@virginia.edu>>

> wrote:

>

>> Given this code:

>>

>> xquery version "1.0-ml";

>> declare option xdmp:output "indent-untyped=yes"; declare option

>> xdmp:output "omit-xml-declaration=yes"; xdmp:document-insert(

>>"/test.xml",

>>

>>  line 1

>>  line2

>>

>> ) ;

>>

>> doc("/test.xml")

>>

>> The output I'm getting is

>>

>>  line

>> 1line2

>>

>> which is disobeying both serialization optinons I specified. ML

>> version 7.0-4.

>> Are other people seeing this? Am I missing something?

>>

>> David

>>

>

>

> Two things:

> (1) If you are running this in QConsole all bets are off, because the

> serialization options apply in the query that does the serialization

> and everything in QConsole is run in a separate eval.

>

> (2) The serialization options are before the semi-colon, that is, they

> only apply to the xdmp:document-insert part of this file. If you want

> them to apply to the serialization of doc("/test.xml") you need to put

> them into that query.

>

> //Mary

> ___

> General mailing list

> General@developer.marklogic.com

> http://developer.marklogic.com/mailman/listinfo/general

>



--

David Sewell, Editorial and Technical Manager ROTUNDA, The University of 
Virginia Press PO Box 400314, Charlottesville, VA 22904-4314 USA

Email: dsew...@virginia.edu   Tel: +1 434 924 9973

Web: http://rotunda.upress.virginia.edu/

___

General mailing list

General@developer.marklogic.com

http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] xdmp:output serialization options not working?

[David Sewell]

Let me restate the problem.

If I use an editor to manually create this XML document and save it to 
MarkLogic 
via WebDav or other means:



   line 1
   line2


then when I subsequently open that document from the database via WebDav or a 
doc() call, it arrives with its indentation preserved. I want to know how to 
generate a new file via XQuery such that it will have indentation when 
subsquently opened via WebDav or a doc() call.

(It's not mission-critical, but I'm auto-generating a 50MB data file that I 
would like to be able to open in oXygen via WebDav with its elements already 
indented, since thing go much faster that way.)

DAvid

On Tue, 14 Oct 2014, Mary Holstege wrote:

> On Tue, 14 Oct 2014 12:28:52 -0700, David Sewell 
> wrote:
>
>> Given this code:
>>
>> xquery version "1.0-ml";
>> declare option xdmp:output "indent-untyped=yes";
>> declare option xdmp:output "omit-xml-declaration=yes";
>> xdmp:document-insert(
>>"/test.xml",
>>
>>  line 1
>>  line2
>>
>> ) ;
>>
>> doc("/test.xml")
>>
>> The output I'm getting is
>>
>> 
>> line 1line2
>>
>> which is disobeying both serialization optinons I specified. ML version
>> 7.0-4.
>> Are other people seeing this? Am I missing something?
>>
>> David
>>
>
>
> Two things:
> (1) If you are running this in QConsole all bets are off, because the
> serialization
> options apply in the query that does the serialization and everything in
> QConsole
> is run in a separate eval.
>
> (2) The serialization options are before the semi-colon, that is, they
> only apply to
> the xdmp:document-insert part of this file. If you want them to apply to
> the
> serialization of doc("/test.xml") you need to put them into that query.
>
> //Mary
> ___
> General mailing list
> General@developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general
>

-- 
David Sewell, Editorial and Technical Manager
ROTUNDA, The University of Virginia Press
PO Box 400314, Charlottesville, VA 22904-4314 USA
Email: dsew...@virginia.edu   Tel: +1 434 924 9973
Web: http://rotunda.upress.virginia.edu/
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] xdmp:output serialization options not working?

[Mary Holstege]

On Tue, 14 Oct 2014 12:28:52 -0700, David Sewell   
wrote:

> Given this code:
>
> xquery version "1.0-ml";
> declare option xdmp:output "indent-untyped=yes";
> declare option xdmp:output "omit-xml-declaration=yes";
> xdmp:document-insert(
>"/test.xml",
>
>  line 1
>  line2
>
> ) ;
>
> doc("/test.xml")
>
> The output I'm getting is
>
> 
> line 1line2
>
> which is disobeying both serialization optinons I specified. ML version  
> 7.0-4.
> Are other people seeing this? Am I missing something?
>
> David
>


Two things:
(1) If you are running this in QConsole all bets are off, because the  
serialization
options apply in the query that does the serialization and everything in  
QConsole
is run in a separate eval.

(2) The serialization options are before the semi-colon, that is, they  
only apply to
the xdmp:document-insert part of this file. If you want them to apply to  
the
serialization of doc("/test.xml") you need to put them into that query.

//Mary
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

[MarkLogic Dev General] xdmp:output serialization options not working?

[David Sewell]

Given this code:

xquery version "1.0-ml";
declare option xdmp:output "indent-untyped=yes";
declare option xdmp:output "omit-xml-declaration=yes";
xdmp:document-insert(
   "/test.xml",
   
 line 1
 line2
   
) ;

doc("/test.xml")

The output I'm getting is


line 1line2

which is disobeying both serialization optinons I specified. ML version 7.0-4. 
Are other people seeing this? Am I missing something?

David

-- 
David Sewell, Editorial and Technical Manager
ROTUNDA, The University of Virginia Press
PO Box 400314, Charlottesville, VA 22904-4314 USA
Email: dsew...@virginia.edu   Tel: +1 434 924 9973
Web: http://rotunda.upress.virginia.edu/
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] How to optimize the REST API Bulk Ingestion Performance?

[Gary Russo]

Hello Danny,

 

Yes, I'm using 7.0-4.

 

>> What are you comparing it to on the Oracle side?

>> In MarkLogic, the content will be all indexed and searchable.  Is that
true on the orcl side too

 

The Oracle side is doing a basic CLOB insert with no indexing.

 

The Oracle server being compared to is a higher capacity system so we
expected to see a faster ingestion.

 

I didn't expect the MarkLogic side to be 4 times slower.

 

Yes, we tried tweaking the batch size. The 500 batch size had the fastest
load times.

 

I will investigate further but I believe the bottleneck is on the MarkLogic
side.

 

I believe the MarkLogic CPU has some room for parallelizing.

 

I'll create a custom REST Extension that will spawn multiple threads for the
doc-inserts.

 

I assume the REST API bulk ingestion already does this but I can't say for
sure.

 

I'll keep you posted.

 

Thanks Danny

 

-  Gary R

 

 

 

From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Danny Sokolsky
Sent: Tuesday, October 14, 2014 2:00 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] How to optimize the REST API Bulk
Ingestion Performance?

 

Hi Gary,

 

A few thoughts here.  You are using 7.0-4 on this?  

 

What are you comparing it to on the Oracle side?  In MarkLogic, the content
will be all indexed and searchable.  Is that true on the orcl side too?

 

What indexes to you have enabled?  Maybe you do not need them all (or maybe
you should put the equivalent indexing on the orcl side)?

 

Have you tried tweaking the batch size?  I would try a smaller number, say
50 or 100.

 

Have you analyzed where you are spending the time?  In the c# code?  In the
code loading the doc on MarkLogic?

 

Do you have multiple threads loading from your .net program?  If you are not
maxing out your cpu on the MarkLogic side, you probably have room for more
parallelization.

 

-Danny

 

From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Gary Russo
Sent: Tuesday, October 14, 2014 9:21 AM
To: general@developer.marklogic.com
Subject: [MarkLogic Dev General] How to optimize the REST API Bulk Ingestion
Performance?

 

MarkLogic Bulk ingestion processing is slower than an equivalent Oracle
ingestion process.

 

The MarkLogic ingestion takes 30 minutes. An Oracle equivalent only takes 7
minutes.

 

I'm using the REST API to bulk ingest multiple documents as described here.
=> http://docs.marklogic.com/guide/rest-dev/bulk#id_54649

 

Notes:

. C# code is used to call the MarkLogic Bulk Ingest REST API.

. Document batch size used is 500.

. Average doc size is 1 KB.

. JSON Conversion and Validation logic occurs in the C# code.

 

 

Any thoughts on how to optimize the MarkLogic bulk ingest to make it as fast
as Oracle's 7 minute load time?

 

 

Thanks,

Gary R

 

 

Gary Russo

Enterprise NoSQL Developer

http://garyrusso.wordpress.com

 

___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Adding new fields

[Danny Sokolsky]

One other thing here:  if you have the reindexer off, you can ask the system 
how many fragments need reindexing using the “preview-reindexer” option in 
xdmp:forest-counts.  For example:

xdmp:forest-counts(xdmp:database-forests(xdmp:database("Documents")), (), 
"preview-reindexer")

This will query the database and calculate the number of fragments needing 
reindexing, returning a report.

-Danny

From: Danny Sokolsky
Sent: Tuesday, October 14, 2014 11:14 AM
To: 'vi...@tilaton.fi'; 'MarkLogic Developer Discussion'
Subject: RE: [MarkLogic Dev General] Adding new fields

You can turn reindexing off during peak times to minimize the impact.  It will 
pick up where you left off when you turn it back on.  Another thing you can do 
is to leave reindexing off, but just rewrite the documents that you want to 
(for example, do a document-insert of a document with its previous content as 
what to insert)—that will have the effect of reindexing just those documents.

If you use the path to specify your field, you can use any path that returns 
true from cts:valid-index-path:

http://docs.marklogic.com/cts:valid-index-path

I am not really understanding how your field can affect every document but you 
only want it in some of the documents.  Maybe the field is not selective enough 
(the path field might help there)?

Also, there are several bug fixes in 7.0-4 wrt fields, so planning and testing 
an upgrade might be a good idea.

-Danny


From: general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] On Behalf Of vi...@tilaton.fi
Sent: Monday, October 13, 2014 11:43 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Adding new fields

Hi,

The indexes do not include the root element. Unfortunately we need to index an 
element that exists in all documents - the only thing that differs is the 
attribute value. (The field index settings are tweaked for specific purposes in 
each case, result from having different full text search requirements for 
specific document sets.)

As this is built on top of another product, we need to have the element named 
like it is in there, and the element is found in all documents. When I look at 
the database status right after adding one such field, I can see that the 
forests are all reindexing totalling millions of docs to go. With new tiered 
hardware this is completed in order of hours, sometimes takes over a day 
though, and with old hardware it took in order of weeks to add one. Our 
monitoring also reveals that it really spikes the usable disk bandwidth, so it 
is definitely working a lot. (My guess is that it selects all the docs with the 
element, but is not intelligent enough to limit using the attribute value too.)

Indexes that include only elements that can be found from a fraction of 
documents are not a problem. Is there some indexing option that I can turn on 
so that ML can index only the docs that have a specific attribute value in the 
given element? Now it seems only capable of querying the docs that have the 
element

This may also be a design issue, but unfortunately I'm unable to do any big 
changes to the way we do things in the codebase I've inherited.

We're running 7.0-2.3 btw, if that matters.

Ville

-- Original Message --
From: "Danny Sokolsky" 
mailto:danny.sokol...@marklogic.com>>
To: "MarkLogic Developer Discussion" 
mailto:general@developer.marklogic.com>>
Sent: 14.10.2014 0:41:51
Subject: Re: [MarkLogic Dev General] Adding new fields

Hi Ville,

I don’t know of a way to tell MarkLogic to trust you in this case, and you 
should not need it to.  If you do not have any content to reindex, and if 
reindexing is enabled, it should not rewrite all of the content.  It will query 
all of the content to see if it needs reindexing, which will not be free but 
should not be too expensive, but I would not expect a full reindex to happen.  
In that case you should see some messages in the log about reindexing that 
database and a little later another message saying you reindexes 0 fragments 
(in fact, you will see these messages each time the config files change).

You mention your fields are doing includes.  I would recommend using paths for 
your fields instead.  Also, make sure your fields are not including the root, 
as that is almost never the correct way to do it.  Are you using 7.0-4 for 
this?  If not, try upgrading.

-Danny

From: 
general-boun...@developer.marklogic.com
 
[mailto:general-boun...@developer.marklogic.com]
 On Behalf Of vi...@tilaton.fi
Sent: Monday, October 13, 2014 12:58 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Adding new fields

Hi,

when developing applications with ML as the database, we need to add new 
indexes regularly to deliver new features. We often (probably 95%) of the time 
add new indexes that will not hit any cont

Re: [MarkLogic Dev General] Adding new fields

[Danny Sokolsky]

You can turn reindexing off during peak times to minimize the impact.  It will 
pick up where you left off when you turn it back on.  Another thing you can do 
is to leave reindexing off, but just rewrite the documents that you want to 
(for example, do a document-insert of a document with its previous content as 
what to insert)—that will have the effect of reindexing just those documents.

If you use the path to specify your field, you can use any path that returns 
true from cts:valid-index-path:

http://docs.marklogic.com/cts:valid-index-path

I am not really understanding how your field can affect every document but you 
only want it in some of the documents.  Maybe the field is not selective enough 
(the path field might help there)?

Also, there are several bug fixes in 7.0-4 wrt fields, so planning and testing 
an upgrade might be a good idea.

-Danny


From: general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] On Behalf Of vi...@tilaton.fi
Sent: Monday, October 13, 2014 11:43 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Adding new fields

Hi,

The indexes do not include the root element. Unfortunately we need to index an 
element that exists in all documents - the only thing that differs is the 
attribute value. (The field index settings are tweaked for specific purposes in 
each case, result from having different full text search requirements for 
specific document sets.)

As this is built on top of another product, we need to have the element named 
like it is in there, and the element is found in all documents. When I look at 
the database status right after adding one such field, I can see that the 
forests are all reindexing totalling millions of docs to go. With new tiered 
hardware this is completed in order of hours, sometimes takes over a day 
though, and with old hardware it took in order of weeks to add one. Our 
monitoring also reveals that it really spikes the usable disk bandwidth, so it 
is definitely working a lot. (My guess is that it selects all the docs with the 
element, but is not intelligent enough to limit using the attribute value too.)

Indexes that include only elements that can be found from a fraction of 
documents are not a problem. Is there some indexing option that I can turn on 
so that ML can index only the docs that have a specific attribute value in the 
given element? Now it seems only capable of querying the docs that have the 
element

This may also be a design issue, but unfortunately I'm unable to do any big 
changes to the way we do things in the codebase I've inherited.

We're running 7.0-2.3 btw, if that matters.

Ville

-- Original Message --
From: "Danny Sokolsky" 
mailto:danny.sokol...@marklogic.com>>
To: "MarkLogic Developer Discussion" 
mailto:general@developer.marklogic.com>>
Sent: 14.10.2014 0:41:51
Subject: Re: [MarkLogic Dev General] Adding new fields

Hi Ville,

I don’t know of a way to tell MarkLogic to trust you in this case, and you 
should not need it to.  If you do not have any content to reindex, and if 
reindexing is enabled, it should not rewrite all of the content.  It will query 
all of the content to see if it needs reindexing, which will not be free but 
should not be too expensive, but I would not expect a full reindex to happen.  
In that case you should see some messages in the log about reindexing that 
database and a little later another message saying you reindexes 0 fragments 
(in fact, you will see these messages each time the config files change).

You mention your fields are doing includes.  I would recommend using paths for 
your fields instead.  Also, make sure your fields are not including the root, 
as that is almost never the correct way to do it.  Are you using 7.0-4 for 
this?  If not, try upgrading.

-Danny

From: 
general-boun...@developer.marklogic.com
 
[mailto:general-boun...@developer.marklogic.com]
 On Behalf Of vi...@tilaton.fi
Sent: Monday, October 13, 2014 12:58 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Adding new fields

Hi,

when developing applications with ML as the database, we need to add new 
indexes regularly to deliver new features. We often (probably 95%) of the time 
add new indexes that will not hit any content in the database currently, but 
know that eventually will when new content is added.

As we have terabytes / millions of docs of content, these reindex operations 
can be costly and take considerable time to run.

So finally to the question: given that we're adding a new field that has one 
include, it seems that ML goes through all documents in the database (include 
limits by element and attribute value) - is there a way to tell ML that hey, we 
know, and we take the responsibility, that the database currently does not have 
any content that needs to be reindex, so 

Re: [MarkLogic Dev General] How to optimize the REST API Bulk Ingestion Performance?

[Danny Sokolsky]

Hi Gary,

A few thoughts here.  You are using 7.0-4 on this?

What are you comparing it to on the Oracle side?  In MarkLogic, the content 
will be all indexed and searchable.  Is that true on the orcl side too?

What indexes to you have enabled?  Maybe you do not need them all (or maybe you 
should put the equivalent indexing on the orcl side)?

Have you tried tweaking the batch size?  I would try a smaller number, say 50 
or 100.

Have you analyzed where you are spending the time?  In the c# code?  In the 
code loading the doc on MarkLogic?

Do you have multiple threads loading from your .net program?  If you are not 
maxing out your cpu on the MarkLogic side, you probably have room for more 
parallelization.

-Danny

From: general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Gary Russo
Sent: Tuesday, October 14, 2014 9:21 AM
To: general@developer.marklogic.com
Subject: [MarkLogic Dev General] How to optimize the REST API Bulk Ingestion 
Performance?

MarkLogic Bulk ingestion processing is slower than an equivalent Oracle 
ingestion process.

The MarkLogic ingestion takes 30 minutes. An Oracle equivalent only takes 7 
minutes.

I'm using the REST API to bulk ingest multiple documents as described here. => 
http://docs.marklogic.com/guide/rest-dev/bulk#id_54649

Notes:

* C# code is used to call the MarkLogic Bulk Ingest REST API.

* Document batch size used is 500.

* Average doc size is 1 KB.

* JSON Conversion and Validation logic occurs in the C# code.


Any thoughts on how to optimize the MarkLogic bulk ingest to make it as fast as 
Oracle's 7 minute load time?


Thanks,
Gary R


Gary Russo
Enterprise NoSQL Developer
http://garyrusso.wordpress.com

___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] How to optimize the REST API Bulk Ingestion Performance?

[Paul M]

The underlying hardware (disk, volume, logical, striping, pipe) is comparable? 
Other than bulk ingest, ML speed is acceptable?




 From: "general-requ...@developer.marklogic.com" 

To: general@developer.marklogic.com 
Sent: Tuesday, October 14, 2014 12:58 PM
Subject: General Digest, Vol 124, Issue 30
 

Send General mailing list submissions to
general@developer.marklogic.com

To subscribe or unsubscribe via the World Wide Web, visit
http://developer.marklogic.com/mailman/listinfo/general
or, via email, send a message with subject or body 'help' to
general-requ...@developer.marklogic.com

You can reach the person managing the list at
general-ow...@developer.marklogic.com

When replying, please edit your Subject line so it is more specific
than "Re: Contents of General digest..."


Today's Topics:

   1. Determining ID of database on anothercluster (Danny Sinang)
   2. Re: How to call xdmp:eval() withtransaction-id option?
  (Gary Russo)
   3. How to optimize the REST API Bulk IngestionPerformance?
  (Gary Russo)
   4. Re: Determining ID of database on anothercluster (Danny Sinang)


--

Message: 1
Date: Tue, 14 Oct 2014 11:16:51 -0400
From: Danny Sinang 
Subject: [MarkLogic Dev General] Determining ID of database on another
cluster
To: general 
Message-ID:

Content-Type: text/plain; charset="utf-8"

I'm trying to write an XQUERY script to configure our production cluster
(running ML 7.0-3) to replicate some of its databases to a new dev cluster.

My plan is to loop through the names of the production databases that I
want replicated and call :

*1. admin:database-foreign-replica($foreign-cluster-id, $foreign-db-id) *and

*2. admin:database-set-foreign-replicas () *

for each database.

Question is, how do I determine *$foreign-db-id* at runtime ? I know
before-hand its name (the same as the production database name), but I
can't seem to find the right function to use to  get the ID of that
"foreign" database.

Any ideas ?

There is of course the option of me running a script on the dev cluster to
generated the db IDs and copying them over to my master cluster script as
hardcoded values, but I'm trying to avoid hardcoding as much as possible.

Regards,
Danny
-- next part --
An HTML attachment was scrubbed...
URL: 
http://developer.marklogic.com/pipermail/general/attachments/20141014/2c2eb45a/attachment-0001.html
 

--

Message: 2
Date: Tue, 14 Oct 2014 11:48:33 -0400
From: Gary Russo 
Subject: Re: [MarkLogic Dev General] How to call xdmp:eval() with
transaction-id option?
To: "'MarkLogic Developer Discussion'"

Message-ID: 
Content-Type: text/plain; charset="us-ascii"

Thanks John



I'm using xdmp:transaction-create() with the update mode option.



xdmp:transaction-create(

  

update

  





I'm using this xdmp:eval() with the transaction-id option.



xdmp:eval(

  $evalCmd,

  (xs:QName("uri"), $uri),

  

{$longTxId}

  





I posted the REST Extension Code here. =>
https://github.com/garyrusso/GLM-Search



I also posted a C# tool that I use to call the REST Extension APIs to test
the ACID rollbacks. => https://github.com/garyrusso/ACIDTester







-Original Message-
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of John Snelson
Sent: Tuesday, October 07, 2014 5:29 AM
To: general@developer.marklogic.com
Subject: Re: [MarkLogic Dev General] How to call xdmp:eval() with
transaction-id option?



Grep the server Modules and Apps directory to find uses of it. You probably
also want to find out about xdmp:transaction-create() as well.



Maybe it's time we documented this functionality - I've seen lots of people
wanting to use it recently.



John



On 06/10/14 23:55, Gary Russo wrote:

> I'm creating a set of REST extensions that will be used in 

> Mulit-Statement ACID Transactions.

> 

> The underlying code will use xdmp:eval() with the transaction-id option.

> 

> Unfortunately, the transaction-id option is undocumented.

> 

> Can someone please provide an example of using xdmp:eval() with a 

> transaction-id option?

> 

> 

> Here's the RESTful APIs that are being created.

> 

> 1. POST /transaction(:

> Returns a transaction-id. e.g.,  :) 2. GET 

> /inventory?rs:type=artichoke&rs:transId=

> 3. GET /inventory?rs:type=bongo&rs:transId=

> 4. PUT

> /inventory?rs:type=artichoke&rs:transId=&rs:action=decr&rs:qua

> ntity=

> 3

> 5. PUT

> /inventory?rs:type=bongo&rs:transId=&r

Re: [MarkLogic Dev General] Determining ID of database on another cluster

[Danny Sinang]

Figured it out.

Had to call xdmp:foreign-cluster-status() and extract the database id from
the list of databases it returns.

Code below is based mostly from
*/opt/MarkLogic/Admin/lib/dbrep-configure-2-form.xqy* .

Regards,
Danny

==

xquery version "1.0-ml";

import module namespace admin = "http://marklogic.com/xdmp/admin";
  at "/MarkLogic/admin.xqy";

declare namespace hs="http://marklogic.com/xdmp/status/host";;
declare namespace fc="http://marklogic.com/xdmp/status/foreign-cluster";;

declare function local:get-local-bootstrap-host($config){
let $bootstrap-hosts := admin:cluster-get-xdqp-bootstrap-hosts($config)
return
if(fn:count($bootstrap-hosts) eq 0) then
fn:error((),"ADMIN-NOBOOTSTRAPHOSTCONFIGURED",())
else
let $first-available-host :=
(xdmp:host-status($bootstrap-hosts)[fn:not(fn:exists(hs:error))]/hs:host-id)[1]
return
if(fn:exists($first-available-host)) then
   $first-available-host
else
fn:error((),"ADMIN-NOBOOTSTRAPHOSTONLINE",())
};

let $config := admin:get-configuration()
let $foreign-cluster-id := admin:cluster-get-foreign-cluster-id($config,
"my-dev-cluster")
let $host-id := local:get-local-bootstrap-host($config)
let $fc-status := xdmp:foreign-cluster-status($host-id,
xs:unsignedLong($foreign-cluster-id))
return

$fc-status/fc:foreign-databases/fc:foreign-database[fc:foreign-database-name
eq 'my-database']

On Tue, Oct 14, 2014 at 11:16 AM, Danny Sinang  wrote:

> I'm trying to write an XQUERY script to configure our production cluster
> (running ML 7.0-3) to replicate some of its databases to a new dev cluster.
>
> My plan is to loop through the names of the production databases that I
> want replicated and call :
>
> *1. admin:database-foreign-replica($foreign-cluster-id, $foreign-db-id) *
> and
> *2. admin:database-set-foreign-replicas () *
>
> for each database.
>
> Question is, how do I determine *$foreign-db-id* at runtime ? I know
> before-hand its name (the same as the production database name), but I
> can't seem to find the right function to use to  get the ID of that
> "foreign" database.
>
> Any ideas ?
>
> There is of course the option of me running a script on the dev cluster to
> generated the db IDs and copying them over to my master cluster script as
> hardcoded values, but I'm trying to avoid hardcoding as much as possible.
>
> Regards,
> Danny
>
>
>
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

[MarkLogic Dev General] How to optimize the REST API Bulk Ingestion Performance?

[Gary Russo]

MarkLogic Bulk ingestion processing is slower than an equivalent Oracle
ingestion process.

 

The MarkLogic ingestion takes 30 minutes. An Oracle equivalent only takes 7
minutes.

 

I'm using the REST API to bulk ingest multiple documents as described here.
=> http://docs.marklogic.com/guide/rest-dev/bulk#id_54649

 

Notes:

. C# code is used to call the MarkLogic Bulk Ingest REST API.

. Document batch size used is 500.

. Average doc size is 1 KB.

. JSON Conversion and Validation logic occurs in the C# code.

 

 

Any thoughts on how to optimize the MarkLogic bulk ingest to make it as fast
as Oracle's 7 minute load time?

 

 

Thanks,

Gary R

 

 

Gary Russo

Enterprise NoSQL Developer

http://garyrusso.wordpress.com

 

___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] How to call xdmp:eval() with transaction-id option?

[Gary Russo]

Thanks John

 

I'm using xdmp:transaction-create() with the update mode option.

 

xdmp:transaction-create(

  

update

  

 

 

I'm using this xdmp:eval() with the transaction-id option.

 

xdmp:eval(

  $evalCmd,

  (xs:QName("uri"), $uri),

  

{$longTxId}

  

 

 

I posted the REST Extension Code here. =>
https://github.com/garyrusso/GLM-Search

 

I also posted a C# tool that I use to call the REST Extension APIs to test
the ACID rollbacks. => https://github.com/garyrusso/ACIDTester

 

 

 

-Original Message-
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of John Snelson
Sent: Tuesday, October 07, 2014 5:29 AM
To: general@developer.marklogic.com
Subject: Re: [MarkLogic Dev General] How to call xdmp:eval() with
transaction-id option?

 

Grep the server Modules and Apps directory to find uses of it. You probably
also want to find out about xdmp:transaction-create() as well.

 

Maybe it's time we documented this functionality - I've seen lots of people
wanting to use it recently.

 

John

 

On 06/10/14 23:55, Gary Russo wrote:

> I'm creating a set of REST extensions that will be used in 

> Mulit-Statement ACID Transactions.

> 

> The underlying code will use xdmp:eval() with the transaction-id option.

> 

> Unfortunately, the transaction-id option is undocumented.

> 

> Can someone please provide an example of using xdmp:eval() with a 

> transaction-id option?

> 

> 

> Here's the RESTful APIs that are being created.

> 

> 1. POST /transaction(:

> Returns a transaction-id. e.g.,  :) 2. GET 

> /inventory?rs:type=artichoke&rs:transId=

> 3. GET /inventory?rs:type=bongo&rs:transId=

> 4. PUT

> /inventory?rs:type=artichoke&rs:transId=&rs:action=decr&rs:qua

> ntity=

> 3

> 5. PUT

> /inventory?rs:type=bongo&rs:transId=&rs:action=decr&rs:quantity=1

> 6. POST /order?rs:transId=(: Prepares Document

> Insert :)

> 7. PUT /transaction?rs:transId=   (: commits transaction
:)

> 8. DELETE /transaction?rs:transId= (: rolls back transaction
:)

> 

> 

> Thanks in advance.

> 

> - GR

> 

> 

> Gary Russo

> Enterprise NoSQL Developer

>   http://garyrusso.wordpress.com

> 

> 

> 

> ___

> General mailing list

>   General@developer.marklogic.com

>  
http://developer.marklogic.com/mailman/listinfo/general

> 

 

 

-- 

John Snelson, Lead Engineer 
http://twitter.com/jpcs

MarkLogic Corporation  
http://www.marklogic.com

___

General mailing list

  General@developer.marklogic.com

 
http://developer.marklogic.com/mailman/listinfo/general

___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general

[MarkLogic Dev General] Determining ID of database on another cluster

[Danny Sinang]

I'm trying to write an XQUERY script to configure our production cluster
(running ML 7.0-3) to replicate some of its databases to a new dev cluster.

My plan is to loop through the names of the production databases that I
want replicated and call :

*1. admin:database-foreign-replica($foreign-cluster-id, $foreign-db-id) *and

*2. admin:database-set-foreign-replicas () *

for each database.

Question is, how do I determine *$foreign-db-id* at runtime ? I know
before-hand its name (the same as the production database name), but I
can't seem to find the right function to use to  get the ID of that
"foreign" database.

Any ideas ?

There is of course the option of me running a script on the dev cluster to
generated the db IDs and copying them over to my master cluster script as
hardcoded values, but I'm trying to avoid hardcoding as much as possible.

Regards,
Danny
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general