[MarkLogic Dev General] Issues with MarkLogic Function in SPARQL Queries in Query Console

[Steve Anderson]

I'm trying to use MarkLogic functions in SPARQL queries.   According to
https://docs.marklogic.com/guide/semantics/semantic-searches

"In addition to SPARQL functions, you can use XQuery or JavaScript built-in
functions (for example, functions with the prefix fn, cts, math, or xdmp)
in a SPARQL query where you can use a function, which includes FILTER, BIND,
and the expressions in a SELECT statement."

It appears to work in the query console, but only intermittently.  Here's a
small repeatable test case.

For example, this, based on the example, works

PREFIX cts: 
SELECT ?o
WHERE
{
  ?s ?p ?o
  FILTER (cts:contains(?o, cts:word-query(("sf"
}

But this, also based on the example, does not

SELECT ?o
WHERE
{
  ?s ?p ?o
   FILTER (( ?o, "sf"
) )
}

Replace "sf" with a string of your choosing.

It returns a stack trace with "[1.0-ml] XDMP-UNDFUN: (err:XPST0017)
Undefined function ()"

Stack Trace

At line 5 column 11:
In sem:sparql("SELECT ?oWHERE{ ?s ?p ?o  FILTER
(( ?o, "sf"
))
6. }
7.
At line 4 column 15:
In xdmp:eval("import module namespace amped-qconsole = 
http://marklogic;, (fn:QName("","query"), "SELECT
?oWHERE{ ?s ?p ?o  FILTER (( ?o, "sf"
))
6. }

In the first grouping, the < before http is highlighted on line 5 and in
the second line 4 is highlighted.

I also tried using the fn: namespace:

PREFIX fn: 
SELECT ?o
WHERE
{
  ?s ?p ?o
   FILTER (fn:starts-with( ?o, "sf" ))
}

And that returns "[1.0-ml] XDMP-UNDFUN: (err:XPST0017) Undefined function
starts-with()"   I've also tired fn:contains with the same result.

This is on Marklogic 8.0-6.4, using developer license, but I also
reproduced it on 9.0-1.1 with an enterprise license.

And yes, before anyone tells me to do this a different way, I know I can do
this in different ways in SPARQL.  I just created a small repeatable query
that illustrates the issue I'm having.

I'm sure I'm doing something wrong, but I'm not sure what.  Can you tell me
what I'm doing wrong?

Thanks,

  Steve
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

[MarkLogic Dev General] Make string XML "safe" in xquery

[Steve Anderson]

I have a string like this:

A title for  the product

and I'd like to replace it with

A title for placeholder the product

Basically, I want to make the a valid XML text node, fixing greater than,
less than, and ampersands.  I thought I could make xdmp:quote do that, but,
perhaps because it's Friday afternoon, I can't find the right options to
make it work.

Is there any easy solution I can't find?

   Steve
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

[MarkLogic Dev General] MarkLogic AMIs and Ansible?

[Steve Anderson]

Before I dig in to the topic, has anyone tried to use ansible to manage
MarkLogic AMIs on AWS?

  Steve
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

[MarkLogic Dev General] Is there a way to delete a stand-alone properties document?

[Steve Anderson]

I'm doing some work with stand-along properties documents, and I want to be
able to delete them.  I know I can delete all the properties with a

xdmp:document-set-properties($uri, ())

but that leaves an empty properties document hanging around.  Is there a
way to delete it?

Thanks,

  Steve
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Search constrained by mime-type?

[Steve Anderson]

Thans y'all!

On Wed, Sep 7, 2016 at 8:08 AM Jason Hunter <jason.hun...@marklogic.com>
wrote:

> I feel compelled to add: MarkLogic 8 also added cts:true-query() for those
> who haven't learned the trick that an AND of empty means all.
>
> cts:search(doc(), cts:true-query(), "format-json") finds all JSON documents
> cts:search(doc(), cts:true-query(), "format-xml") finds all XML documents
>
> There's a cts:false-query() too.
>
>
> -jh-
>
> On Sep 7, 2016, at 9:29 PM, Joe Bryan <joe.br...@marklogic.com> wrote:
>
> Hi All,
>
> MarkLogic 8 adds the format-* option to cts:search():
>
> cts:search(/, cts:and-query(()), "format-json") finds all JSON documents
> cts:search(/, cts:and-query(()), "format-xml") finds all XML documents
>
> And so on, for  binary, json, text, or xml.
>
> http://docs.marklogic.com/cts:search?q=cts:search=8.0=true
>
> Thanks.
>
> -jb
>
> From: <general-boun...@developer.marklogic.com> on behalf of Dave Cassel <
> dave.cas...@marklogic.com>
> Reply-To: MarkLogic Developer Discussion <general@developer.marklogic.com>
> Date: Wednesday, September 7, 2016 at 7:26 AM
> To: MarkLogic Developer Discussion <general@developer.marklogic.com>
> Subject: Re: [MarkLogic Dev General] Search constrained by mime-type?
>
> I think you'd need to write a custom constraint
> <http://docs.marklogic.com/guide/search-dev/search-api#id_49750> to do
> that.
>
> --
> Dave Cassel <http://davidcassel.net/>, @dmcassel
> <https://twitter.com/dmcassel>
> Technical Community Manager
> MarkLogic Corporation <http://www.marklogic.com/>
> http://developer.marklogic.com/
>
>
> From: <general-boun...@developer.marklogic.com> on behalf of Steve
> Anderson <steve.ander...@gmail.com>
> Reply-To: MarkLogic Developer Discussion <general@developer.marklogic.com>
> Date: Wednesday, September 7, 2016 at 12:10 AM
> To: MarkLogic Developer Discussion <general@developer.marklogic.com>
> Subject: [MarkLogic Dev General] Search constrained by mime-type?
>
> Is there a way to constrain a search to a certain mime-type?
>
> For example, if I want to search every JSON document, but not any XML
> documents, how can I do that?
>
> Thanks,
>
>   Steve
> ___
> General mailing list
> General@developer.marklogic.com
> Manage your subscription at:
> http://developer.marklogic.com/mailman/listinfo/general
>
>
> ___
> General mailing list
> General@developer.marklogic.com
> Manage your subscription at:
> http://developer.marklogic.com/mailman/listinfo/general
>
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

[MarkLogic Dev General] Search constrained by mime-type?

[Steve Anderson]

Is there a way to constrain a search to a certain mime-type?

For example, if I want to search every JSON document, but not any XML
documents, how can I do that?

Thanks,

  Steve
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load?

[Steve Anderson]

That did it, Erik, thanks.

On Fri, Jul 1, 2016 at 8:57 PM Erik Hennum <erik.hen...@marklogic.com>
wrote:

> Hi, Steve:
>
> Our cookbook examples are maybe a bit misleading in this respect.
>
> Each example creates its own database client merely so it can stand alone.
>
> A realistic application would keep a singleton database client until the
> application completely finishes using the REST server.  To put it the other
> way, there's no reason to close a database client if the application is
> still using the same REST server as the same user.
>
> The connection pool is associated with the database client (and, in fact,
> is Apache HttpClient in the lower layers), so using a singleton database
> client (including sharing across multiple threads) should greatly improve
> the socket efficiency.
>
>
> Hoping that helps (and that things are well with you),
>
>
> Erik Hennum
>
>
> --
> *From:* general-boun...@developer.marklogic.com [
> general-boun...@developer.marklogic.com] on behalf of Steve Anderson [
> steve.ander...@gmail.com]
> *Sent:* Friday, July 01, 2016 5:02 PM
> *To:* MarkLogic Developer Discussion
> *Subject:* Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT
> under load?
>
> Thanks for the fast response, Erik, hope you're doing well.
>
> I am not using a single database client object.  The docs say to close()
> the client after use.  Doing that makes the client unusable for further
> transactions, so I've been creating a new one for each request.  Should I
> have a singleton client and never use close(), or is there some in-between
> best practice??
>
> I believe I am writing and reading to completion.
>
>   Steve
>
> On Fri, Jul 1, 2016 at 4:40 PM Erik Hennum <erik.hen...@marklogic.com>
> wrote:
>
>> Hi, Steve:
>>
>> A few questions:
>>
>> *  Are you creating a single database client object and using it for all
>> requests on the REST port?
>>
>> That's important for connection pooling.  The database client is
>> thread safe after configuration.
>>
>> For instance, in a Java servlet environment, the single database
>> client would be created during
>> application initialization.
>>
>> *  Are you writing and reading to completion?  For instance, closing
>> streams after reading the data.
>>
>>
>> Hoping that helps,
>>
>>
>> Erik Hennum
>>
>>
>> --
>> *From:* general-boun...@developer.marklogic.com [
>> general-boun...@developer.marklogic.com] on behalf of Steve Anderson [
>> steve.ander...@gmail.com]
>> *Sent:* Friday, July 01, 2016 4:09 PM
>> *To:* general@developer.marklogic.com
>> *Subject:* [MarkLogic Dev General] Sockets being left in CLOSE_WAIT
>> under load?
>>
>> I'm using version 3.0.5 of the Java API client on a Mac with Oracle's
>> Java 1.8.0_72 and MarkLogic 8.0-5.1.
>>
>> I've noticed that my app under load (multiple requests to different end
>> points) is leaving of sockets between the server and the database in the
>> CLOSE_WAIT state.  Under load (thousands of document writes in multiple
>> threads), it climbs and climbs and climbs, with no apparent end in sight.
>>
>> If I force a cleanup via the debugger (System.gc();
>> System.runFinalization ();) the majority of the sockets close.
>>
>> When I run AllCookbookExamples, and, in another terminal, running "lsof
>> -i -n -P|grep _WAIT" I see the same thing; the number of sockets left open
>> grows until the session ends.  It's no different with the current develop
>> branch on github.
>>
>> I suspect it's something I'm missing related to connection pooling, but I
>> don't see any documentation on connection pooling configuration.
>>
>> I know I can modify ulimit on my system, but I'm afraid it's an
>> underlying issue in my code that will keep the sockets open forever and
>> eventually cause network issues.
>>
>> Has anyone else seen anything like this?
>>
>>   Steve
>>
>> ___
>> General mailing list
>> General@developer.marklogic.com
>> Manage your subscription at:
>> http://developer.marklogic.com/mailman/listinfo/general
>>
> ___
> General mailing list
> General@developer.marklogic.com
> Manage your subscription at:
> http://developer.marklogic.com/mailman/listinfo/general
>
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load?

[Steve Anderson]

Thanks for the fast response, Erik, hope you're doing well.

I am not using a single database client object.  The docs say to close()
the client after use.  Doing that makes the client unusable for further
transactions, so I've been creating a new one for each request.  Should I
have a singleton client and never use close(), or is there some in-between
best practice??

I believe I am writing and reading to completion.

  Steve

On Fri, Jul 1, 2016 at 4:40 PM Erik Hennum <erik.hen...@marklogic.com>
wrote:

> Hi, Steve:
>
> A few questions:
>
> *  Are you creating a single database client object and using it for all
> requests on the REST port?
>
> That's important for connection pooling.  The database client is
> thread safe after configuration.
>
> For instance, in a Java servlet environment, the single database
> client would be created during
> application initialization.
>
> *  Are you writing and reading to completion?  For instance, closing
> streams after reading the data.
>
>
> Hoping that helps,
>
>
> Erik Hennum
>
>
> --
> *From:* general-boun...@developer.marklogic.com [
> general-boun...@developer.marklogic.com] on behalf of Steve Anderson [
> steve.ander...@gmail.com]
> *Sent:* Friday, July 01, 2016 4:09 PM
> *To:* general@developer.marklogic.com
> *Subject:* [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under
> load?
>
> I'm using version 3.0.5 of the Java API client on a Mac with Oracle's Java
> 1.8.0_72 and MarkLogic 8.0-5.1.
>
> I've noticed that my app under load (multiple requests to different end
> points) is leaving of sockets between the server and the database in the
> CLOSE_WAIT state.  Under load (thousands of document writes in multiple
> threads), it climbs and climbs and climbs, with no apparent end in sight.
>
> If I force a cleanup via the debugger (System.gc(); System.runFinalization
> ();) the majority of the sockets close.
>
> When I run AllCookbookExamples, and, in another terminal, running "lsof -i
> -n -P|grep _WAIT" I see the same thing; the number of sockets left open
> grows until the session ends.  It's no different with the current develop
> branch on github.
>
> I suspect it's something I'm missing related to connection pooling, but I
> don't see any documentation on connection pooling configuration.
>
> I know I can modify ulimit on my system, but I'm afraid it's an underlying
> issue in my code that will keep the sockets open forever and eventually
> cause network issues.
>
> Has anyone else seen anything like this?
>
>   Steve
>
> ___
> General mailing list
> General@developer.marklogic.com
> Manage your subscription at:
> http://developer.marklogic.com/mailman/listinfo/general
>
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

[MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load?

[Steve Anderson]

I'm using version 3.0.5 of the Java API client on a Mac with Oracle's Java
1.8.0_72 and MarkLogic 8.0-5.1.

I've noticed that my app under load (multiple requests to different end
points) is leaving of sockets between the server and the database in the
CLOSE_WAIT state.  Under load (thousands of document writes in multiple
threads), it climbs and climbs and climbs, with no apparent end in sight.

If I force a cleanup via the debugger (System.gc(); System.runFinalization
();) the majority of the sockets close.

When I run AllCookbookExamples, and, in another terminal, running "lsof -i
-n -P|grep _WAIT" I see the same thing; the number of sockets left open
grows until the session ends.  It's no different with the current develop
branch on github.

I suspect it's something I'm missing related to connection pooling, but I
don't see any documentation on connection pooling configuration.

I know I can modify ulimit on my system, but I'm afraid it's an underlying
issue in my code that will keep the sockets open forever and eventually
cause network issues.

Has anyone else seen anything like this?

  Steve
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

[MarkLogic Dev General] Java Client API: Setting the "store" for a SPARQL executeSelect?

[Steve Anderson]

I'm using triples in my document properties, and my queries work great when
I'm working with the query console, either via a SPARQL query or a XQuery
with sem:sparql.

When I query with the Java API, though, I don't get any results for triples
stored in properties, only in documents.

My query is:

String sparqlSelect = "SELECT (COUNT(?p) as ?pCount) WHERE { ?s ?p ?o .}"
SPARQLQueryDefinition qdef1 = sparqlMgr.newQueryDefinition(sparqlSelect)
JsonNode jsonStrResults = sparqlMgr.executeSelect(qdef1, new
JacksonHandle()).get()

That returns a much smaller number then I expect.

I can reproduce the exact results with XQuery by setting the sem:store
option to "documents"

xquery version "1.0-ml";
import module namespace sem = "http://marklogic.com/semantics; at
"/MarkLogic/semantics.xqy";
let $query := "SELECT (COUNT(?p) as ?pCount) WHERE { ?s ?p ?o .}"
return sem:sparql($query, (), (), sem:store("document"))

If I remove all the options from the sem:sparql, or change sem:store to
"any" I get a larger number, one that includes the unmanaged triples stored
in properties.

Can anyone tell me how to set the equivalent in the Java API so that I can
search all the triples, not just those in the documents?  I'm open to
making database configuration changes if that's required.

Thanks,

  Steve
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general