Hi Erik/Florent
Thanks a lot for response and suggestions.
We are already using compiled schematron (XSLT) and schema (XSD) till now. we
recently started using schemas from IMS Global and all of these are with
embedded schematron.
As the schema is always passing we thought all looks good but
Thanks Florent Georges
It helps a lot as we can extract the schematron and compile and use it.
I am looking for any out of the box solution which can do it without extracting
the schematron from schema. I understand schema and schematron are 2 different
concepts but as we are now having
HI Christopher
If a schema is having a schematron with in it. Is there any way we can validate
both schema and schematron rules against an XML document at once?
Schema validation works fine to check the structure of the document but the
inbuild schematron holds the data level complex
HI Christopher
If a schema is having a schematron with in it. Is there any way we can validate
both schema and schematron rules against an XML document?
Schema validation works fine to check the structure of the document but the
inbuild schematron holds the data level complex validations which
Hi
Is there any way to spawn threads on all hosts in a cluster instead of one host?
As I want to spawn more threads a schedule tasks which will be spawning huge
amount of threads (very light weight) I am checking if we can spawn across
cluster to ensure we don't have issues with time out or
Hi
I am just wondering is there any way we have JSON documents with namespaces.
When an XML document is converted as JSON we lose the namespaces as JSON won't
carry namespace
and we try to covert JSON back to XML, the XML no longer carry the namespace as
the source is JSON.
Is there any
Hi Shahidhar
Yes, MarkLogic will re-index for the removed fields.
You might feel that it got stuck at49.6% but ML would be merging in background
to ensure we have less stands.
By now it should have completed the merging and re-indexing reclaiming your
disk space.
Regards,
Gnana(GP)
Date:
Hi Erik
I missed some of your questions.
Yes, I am persisting the search option in REST DB and using it.
The same one when I try in QC the debug is showing my additional constraint as
below
search:report id=SEARCH-FLWOR(cts:search(fn:collection(),
cts:query(lt;cts:and-query
Hi Erik
I update my search option to debug this and the result is as below
Search Options:
search:options xmlns:search=http://marklogic.com/appservices/search;
search:quality-weight0/search:quality-weight
search:search-optionunfiltered/search:search-option
Hi
I am facing an issue with additional-query in REST Search options.
I am having search options as below. I am expecting the results only with
element hmhGuid.
search:options xmlns:search=http://marklogic.com/appservices/search;
search:quality-weight0/search:quality-weight
Hi
I am trying to search for documents with few constraints (indexes created).
I am having 1000's of documents in my system which are as per my search
criteria.
Now, I want to have an additional constraint which should return docs with a
specified element in them (hmhGuid).
Note: this is
HI Josten/Erik
Our cluster configuration is as below
3 Nodes Cluster with 1 D-Node and 2 e-Nodes
All the content, modules, services are in D-Node.
We are using D-Node also for now as e-Node.
Using our build we are registering our Rest Resource Extensions on Node-1 where
our services exists.
Hi
I am facing an issue with resource extensions in cluster environment.
We are having 3 node cluster and we are registering our resource extension to
node 1 which is our bootstrap node.
The resource extension is working without issue on node 1 where we registered
but not working as expected
Hi
I am facing an issue with resource extensions in cluster environment.
We are having 3 node cluster and we are registering our resource extension to
node 1 which is our bootstrap node.
The resource extension is working without issue on node 1 where we registered
but not working as expected
Hi Eric
We are storing only XML files at the moment as ML 6 and Below we have indexing
only for XML files.
But end user is sending JSON document (Full ML JSON format) which we are
transforming to XML in the transform module of PUT request.
The XML is validated against schema and then stored
Hi Eric
In ML 6 we(myself and vikas) used to send a request with .xml extension but
pass JSON document.
Within transformation we used to convert JSON to XML, validate and insert the
XML document into the system with the specified URL
Now this is not working in ML 7.
As you mentioned, if we
Thanks for the details Lee/ Michael
I am referring to HTTP App-Server ML REST API.
The tests we are trying to perform is for 400 concurrent users using JMeter. In
the Ramp-up time for 15 minutes we see no issues in the performance.
As soon as the 400 concurrent users are up, we see the threads
Hi
I want to understand how threads in app servers work.
I believe for every request with authentication MarkLogic will show 2 threads.
But what I see is when we are doing performance testing, MarkLogic is showing
32 threads are being used per node (we have 3 nodes in a cluster) but active
Hi David
Thanks for the explanation.
I am currently using JMeter for performance testing.
When I say performance is degraded, the 90th percentile for a request increased
from 0.08 seconds (for 50 users) to 18 seconds (for 400 users).
When I increased number of threads from 32(default) to 100,
Hi
I am facing an issue with performance when we have default 32 threads for our
application server.
We are running performance test with 50/100/400 concurrent users and
performance is degraded a lot when we are running with 400 concurrent users.
By increasing thread count to 100 from 32,
Hi
I got a work around for the issue with JSON and am sharing for our future
reference
I am going through ML 7 documentation and found that we have new search options
now and one among them is result-decorator
https://docs.marklogic.com/guide/search-dev/query-options#id_63172
I used custom
HI
I am trying to search for text with quotes but it's not working as expected.
For example:
program:testing program
Below is the query being constructed which is wrong. Can I know how to search
for string with quotes using REST API?
query: {
and-query: {
Hi Mike
I am using ML 6.0-2.3 version and I tried as you suggested but no change.
Below the stands that are showing up for my forest.
Stand CategoryActive FragmentsNascent Fragments
Deleted Fragments On Disk SizeIn Memory Size
071aActive
Hi
I have been using json api in MarkLogic and now I am experiencing performance
issues when I try to convert large xml response into json.
XML response is taking 3 seconds while converting this xml response to json
using json api is taking 12 seconds on an average.
Below is the simple code
Hi
I am trying to merge a specific forest but it's not performing merge operations
as expected.
Code snippet being used:
xdmp:merge(options xmlns=xdmp:merge
merge-max-size4000/merge-max-size
forests
forest{xdmp:forest(content-1)}/forest
Hi
I am having an issue with path range query. all of a sudden am getting
different results.
declare namespace res = http://www.hmh.com/resource;;
declare variable $collection := Resources;
declare variable $case-insensitive-collation :=
collation=http://marklogic.com/collation//S2;;
declare
Hi
I tried to perform query trace on all these 3 queries.
Interestingly all are selecting one fragment which is expected but the 2nd
query is not giving me count and also when I remove count I am not getting the
document back.
Any valid reasons for this issue?
Regards,
Gnana(GP)
Hi Damon
I am using this particular condition to exclude some of the fragments.
Path range index is not allowing me to use wildcard so I have to supply list of
values instead of specifying a wildcard like *.
Even my stats show when we supply more values it taking more time than expected
or
Hi Abhinav
I do accept that namespace should be used in XPath's for better performance.
The question in context is path range index which I believe have no impact once
the content is indexed with a path range.
May be I am wrong in my above statement. Would love to hear on path range
indexes
Hi
I am profiling one of my queries and found that path range query is taking more
time when multiple values are passed.
Example:
cts:path-range-query(*:resource/*:level/*:level/*:level/@type,=,(module,lesson,unit,day)
,$case-insensitive-collation)
Shallow µs: 600-900
takes more time than
Hi
Is there any way to specify path index as sort order in Search options?
Regards,
Gnanaprakash Bodireddy
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged information.
If you are not the intended
Hi
I am trying to see the best approach to sort the content/results in REST.
We have millions of files in our DB and when we try to use sort field it gives
expand tree cache exception.
What is the best approach to include sorting on a specific field?
Regards,
Gnanaprakash Bodireddy
This
Thanks a lot gary vidal :) that's exactly what I am looking for.
Regards,
Gnana(GP)
Message: 3
Date: Tue, 14 Jan 2014 18:49:05 +
From: Gary Vidal gary.vi...@marklogic.com
Subject: Re: [MarkLogic Dev General] cts query with validation to
check
To: general@developer.marklogic.com
Hi
Is there any way to have a cts query in which I can restrict the scope based on
count?
For example:
book
title/
programs
program/
program/
/programs
/book
I want to remove above document from search
Hi Erik
Using below code is not giving me format=json.
search:snippet
json type=object xmlns=http://marklogic.com/xdmp/json/basic;
key type=stringvalue/key
/json
/search:snippet
Using below code gave me json response in snippet but it is not the best
Hi
I recently updated my MarkLogic server from v6 to v7 and I got an issue with
json conversion.
Details:
MarkLogic 6:
As part of REST-API, when we supply the parameter format=json, the entire
response was getting converted into JSON irrespective of what we have in
content node.
i.e. By
Hi Erik
Yes, it's custom transformation I am using wherein I am converting XML to JSON
to ensure my raw xml will be shown in JSON format.
Everything is working fine in ML6 and in ML7. even though my JSON conversion is
working, the content element is treating the JSON as string and wrapping it
Hi
I am trying to find a way to have a custom transformation for facets similar to
results.
To perform transformation on the results we use below code snippet
search:transform-results apply=raw
ns=http://marklogic.com/rest-api/transform/name at=/name.xqy
/search:transform-results
Is there any
HI
Not sure whether below one is a valid solution or not.
Keeping this in search options ensured we have our search scope limited to
field (include elements)
search:term apply=term
search:default
search:word
search:field name=field-name/
Hi
Is there any way to include/exclude set of elements in a fragment while
searching?
For example:
root
element1/
element2/
...
element10/
/root
While performing search (REST API based), I want my search (keyword or
We initially started with word-lexicons (include/lexicons) only. But I guess
once we configure word lexicons with include and exclude it will impact regular
word searches.
Later we went for fields.
Somehow I am feeling this may not be a best solution as the include/exclude
fields may get
Thanks a lot Damon
That's the best solution and I love this forum as we always get the best
solutions and clarifications.
Regards,
Gnana(GP)
Gnana,
The map is a hash map, so I assume the order of the elements is affected by how
they get allocated to hash buckets. You could try this
Hi
I am experiencing a performance issue with ML 6 REST Instance.
I am trying to perform a simple search on keyword in a specific collection for
1000 concurrent users. But the response time is shooting up to the max
20seconds and min less than a second.
Can you please let me know what's the
Hi
I am just wondering if we could load only JSON documents into MarkLogic and
then index the keys as we do for XML elements? (I know indexing is possible but
my though
Can we create indexes similar to XML elements range, path range indexes?
Can we query the JSON similar to XQuery?
Gnana(GP)
Hi
I am facing an issue with total count and actual result count in REST Based
search.
Total count is showing as 9 while the actual # of results are 10.
I know based on fragments the total count will vary as it uses xdmp:estimate.
Can someone help me out in understanding the fragment count Vs
Hi Erik
I am trying to figure out whether we can have a different constraint name and
facet name?
For example:
search:constraint name=title
search:range collation=http://marklogic.com/collation//S2;
type=xs:string facet=true
search:facet-optionfrequency-order/search:facet-option
Hi
I am currently having word lexicons's configured on a collation (string
collation), through which I can get all the list of words available in a
database (I know that we may have performance impact when we have huge DB).
I am using this for search suggestions with wild card options.
The
Hi Erik,
Thanks for your reply and suggestions :)
I now understand the issue I got into. I updated my search options as filtered
and it's working as expected now.
We will review the performance impact and will revert our configuration
(options) change if required.
Gnana
Gnanaprakash
Hi Tom
I tried the command you gave and its working fine for me
curl -X PUT --anyauth --user admin:admin
http://localhost:8005/v1/documents?uri=/example/recipeformat=json;
-d...@sample.txt
I used this from windows machine and the document created is similar to what
you gave
?xml
Hi Erik
Thanks a ton for your suggestion Suppress the entire label if you don't need
the label in JSON
I tweaked my code to suppress the highlighting tag and show it as text/string
and it worked brilliantly.
Gnana
Gnanaprakash Bodireddy | Sr. Associate - Projects | IME | Cognizant Technology
Hi Erik
Is there any way where we can specify in configurations not to treat an element
for json conversion.
If there is a way then I think we can easily handle the issue I am facing.
Gnana
Gnanaprakash Bodireddy | Sr. Associate - Projects | IME | Cognizant Technology
Solutions.
Address:
Hi Erik,
I am having a requirement to Search based on
1. Rest API Call -- 2. Call an external service -- 3. Get Results --
4. Ingest transformed document -- 5. Perform Rest API Search
Note: the search will be effected drastically based on step 4 (sometimes based
on condition steps 2 to
Hi Erik
I think I didn't explain my requirement well.
Search based on Rest (Get) should have the below steps internally
1. Rest Search -- 2. Call an external service -- 3. Get Results -- 4.
Update document -- 5. Rest Search
The main issue here is we are updating a document
Hi Erik,
Can you please help me out in this. There seems to be an issue in custom
transformation parameter sequence.
Regards,
Gnanaprakash Bodireddy.
From: Bodireddy, Gnanaprakash (Cognizant)
Sent: Thursday, December 27, 2012 11:07 AM
To: general@developer.marklogic.com
Subject: Issue with ML6
Hi Erik
Thanks for your suggestions.
I rolled back my changes in Rest-API and went ahead with custom transformation
to convert my XML into JSON and it worked brilliantly.
Simply loving the way MarkLogic Rest-API JSON API's are working :)
Regards,
Gnanaprakash Bodireddy
From: Erik Hennum
Hi
I am facing an issue in using custom XQuery Transformation which I am currently
using in Rest-API Search
http://Host:9000/v1/search?q=options=levelTransformationhttp://%3c%3cHost%3e%3e:9000/v1/search?q=options=levelTransformation
levelTransformation is configured as below:
options
Hi Charles
All the documents I ingested are XML's not JSON. But as our application is
consuming JSON I have to give the response in JSON which is why I explored XML
to JSON Api.
To summarize,
1. I have a schema defined for just to ensure JSON is handling data types
2. I have a Schematron
Hi Erik
Thanks for the help.
I have created the options and it worked fine for me. But I got results which
is slightly deviating from my expectation.
Options: custom-search
options xmlns=http://marklogic.com/appservices/search;
transform-results apply=raw/
/options
URL:
HI Erik
Can you please help in how I can pass transform-results as options in URL. Do I
need to configure anything before passing transform-results?
The below one is not working for me.
http://Host:9000/v1/search?q=format=xmloptions=transform-results:raw
Thanks and Regards,
Gnanaprakash
Hi Dave
I have already started exploring JSON XQuery library which is not exposed in
REST calls as you mentioned :)
Meanwhile, I tried using schema for handling data types and it's working
perfectly.
Thanks for your suggestion :)
Thanks and Regards,
Gnanaprakash Bodireddy
Message: 3
Date:
Hi Charles Greer,
I inserted a new schema into my Schema database and updated all my content in
my Content database to map schema.
For handling data types, having a schema is the best approach as there won't be
any need to think about new elements being added to the content (we can just
Hi Charles Greer,
I inserted a new schema into my Schema database and updated all my content in
my Content database to map schema.
For handling data types, having a schema is the best approach as there won't be
any need to think about new elements being added to the content (we can just
Hi
I am exploring MarkLogic XML to JSON Rest API and am stuck with how to handle
data types.
For example:
user
id1/id
nameGnana/name
/user
Actual JSON Response:
{user:{id:1, name:Gnana}}
Expected JSON Response:
{user: {id:1, name:Gnana}}
How can I make sure
Hi David
Previously when I used Corona I had my custom json conversion where I am
handling the data types based on the element names. So I thought this may be
available in ML 6 XML to JSON conversion also.
If this is not available. I think it is worth to include :)
Thanks and Regards,
Hi
I need some help in retrieving full document content while performing search
using MarkLogic 6 Rest Services
I have configured my Rest Http server on 9000 port and I am trying to perform a
word-query (term query) and want to get all resultant documents.
By using below query search is
Hi Mike
We got a solution from Norman which improved performance a lot.
The solution we got is to compile the Schematron before in hand and supply it
to Validation which actually reduced the overhead of compiling the Schematron
every time.
Thanks and Regards,
Gnanaprakash Bodireddy
Date:
Hi Mike
Please find my comments below
Which version of MarkLogic are you using? On what OS?
Gnana: MarkLogic 5.0.3, Linux
50 threads could be too many. How many CPU cores does the host have? How much
RAM?
Gnana:
As MarkLogic supports 256 threads I thought 50 threads are good to use. I am
just
Hi
I am currently validating XML's using Schematron. But I am facing a performance
issue with this.
Currently I am generating large number of documents on the fly and using
triggers (50 Threads) to validate the XML's.
Each XML when validated individually is taking around 0.4s but when using
Hi
Is there any way in MarkLogic to Validate a schema (which have Schematron
internally)
Till now,
I have used Schema to validate the XML data and it went on well with commands
like (validate strict xml document and validate full xml document)
After having a requirement of performing
Hi
I am trying to create a schema for an XML as below
level hierarchy=0 type= Level1Type
level hierarchy=1 type=Level2Type 2
level hierarchy=2 type= Level3Type
.
/level
Hi
Is it possible to include conditional validation in Schema.
For example:
Sample1: (Valid)
root
level type=skill
lesson/
/level
/root
Sample2: (Invalid)
root
level type=section
lesson/
/level
/root
In the above sample, I want to
Hi
I am validating the XML's using the schema I stored in the schema database.
Validation works fine. But, I am getting only one error when xml is not valid.
Currently I am looping through all nodes of an XML and validating them with the
XSD which is giving me all errors. But this takes more
Thanks a lot Mary. It worked brilliantly. ☺
Thanks and Regards,
Gnanaprakash Bodireddy
Sr Associate - Projects | IME
Cognizant Technology Solutions
VNET: 672103
(O): +91 (40) 4451 extn: 672103
(M): +91-8897575644
-Original Message-
From: Mary Holstege
Hi
Need help in sorting untypedAtomic Datatypes.
I am facing issue while sorting untypedAtomic types (i.e. combination of
string and integer)
E.g.
let $data := document {
standards
standardSC.3.P/standard
standardSC.3.P.10/standard
standardSC.3.P.10.2/standard
Hi John
The code you gave works but I am afraid of hard coding the index value
Because, the length of the string may vary and the tokens may increase from 5
to 8 or even 10.
E.g.
let $data := document {
standards
standardSC.3.P/standard
standardSC.3.P.10/standard
Hi John/Geert
My data a bit more complex than we are discussing now.
E.g.
let $data := document {
standards
standardSC.3.P/standard
standardSC.3.P.10/standard
standardSC.3.P.10.2/standard
standardSC.3.P.11/standard
standardSC.3.C.11/standard
standardSC.3.P.11.1/standard
Hi,
Can you please help me in getting entity enrichment related
installation.
When I look at the host-stauts-options, I can see entity enrichment.
But, when I perform entity:enrich it giving error as Entity enrichment
libraries not installed for host.
Do I need to install any other
Hi,
I am trying to update corona namespace.
Eg.
I have a namespace created as corona-index-standard which is created
for an index on element standard.
Assume there is an index on element id and I want the namespace
corona-index-standard to refer the index one element id instead of
Hi,
Is there any possibility of storing content in MarkLogic cache with
expiration policy.
For example,
I will get an xml from external server and want to store that in
MarkLogic cache which will expire in 6hrs.
I want to use the xml stored in cache in my XQuery modules and whenever
Hi Geert/Soumadri,
Thanks for the information.
I created my custom collector and am able to chunk my file into smaller files
based on my conditions and insert into the target database using Information
studio.
Thanks and Regards,
Gnanaprakash Bodireddy
Sr Associate - Projects | IME
Cognizant
80 matches
Mail list logo
