Re: [MarkLogic Dev General] Range Index in Where Clause
Thanks for the help!
David
On Oct 22, 2012, at 1:27 PM, Gajanan Chinchwadkar
wrote:
> If an xpath expression has a predicate with value, then element range index
> will be used, if available.
> For example, in the following two cases a range index on xs:QName("title")
> will be used:
> a. /bookstore/book[title="XQuery Programming"]
> b. /bookstore/book[title="XQuery Programming"]/author
>
> However, if there is no value in the predicate then the range index will not
> be used:
> c. . /bookstore/book[title]
>
> The same logic applies to where clause. If there is a value in the where
> clause and a corresponding element range index, it will be used.
> It is different in order by, where an index gets used even though there is no
> comparison predicate and value.
>
> In 6.0.1-1, XPath based index can be defined. There are many situations when
> MarkLogic server attempts to find matching path range index. Again in order
> to use range index, there must be a comparison predicate in the query.
>
> Thanks,
>
> Gajanan
>
> -Original Message-
> From: general-boun...@developer.marklogic.com
> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Michael Blakeley
> Sent: Monday, October 22, 2012 9:47 AM
> To: MarkLogic Developer Discussion
> Subject: Re: [MarkLogic Dev General] Range Index in Where Clause
>
> I would rephrase this as "does the evaluator ever use indexes for where
> clauses?" If indexes are used at all, I would expect ranges to work too. And
> with 6.0-1.1 the answer seems to be "yes" - but I would still recommend doing
> as much as possible in XPath predicates.
>
> Here's a contrived test. In real life it would be very odd to have a
> dls:created element in the main document, but since the range index comes
> preconfigured it's handy for testing.
>
> declare namespace dls="http://marklogic.com/xdmp/dls";;
>
> xdmp:document-insert(
> 'test/1',
> element a {
>element b { 'c' },
>element dls:created { xs:dateTime(xs:date('2012-12-01')) } }),
> xdmp:document-insert(
> 'test/2',
> element a {
>element b { 'c' },
>element dls:created { xs:dateTime(xs:date('2012-12-12')) } }) => ()
>
> Now we have two documents that match the /a[b eq 'c'] portion of the
> expression, but with different dsl:created values. Let's write a test query
> that matches one, but not the other:
>
> declare namespace dls="http://marklogic.com/xdmp/dls";;
> xdmp:query-trace(true()),
> for $n in /a[ b eq 'c' ]
> where $n/dls:created < xs:dateTime(xs:date('2012-12-12'))
> return $n
>
> One document should match the dls:created constraint, and the other
> shouldn't. The actual results match expectations, but that could be due to
> filtering. Let's check the trace:
>
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:11: Analyzing path for $n:
> fn:collection()/a[b eq "c"]
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:11: Step 1 is searchable:
> fn:collection()
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:11: Step 2 is searchable: a[b eq
> "c"]
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:11: Path is fully searchable.
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:11: Gathering constraints.
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:14: Comparison contributed hash
> value constraint: b = "c"
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:11: Step 2 predicate 1
> contributed 1 constraint: b eq "c"
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:14: Comparison contributed hash
> value constraint: b = "c"
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:11: Step 2 predicate 1
> contributed 1 constraint: b eq "c"
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:11: Step 2 contributed 2
> constraints: a[b eq "c"]
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 5:9: Comparison contributed
> dateTime range value constraint: dls:created <
> xs:dateTime("2012-12-12T00:00:00")
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:11: Where clause contributed 1
> constraint for $n: $n/dls:created < xs:dateTime("2012-12-12T00:00:00")
> 2012-10-22 09:06:18.661 Info: 8003-cq: at 4:11: Executing search.
> 2012-10-22 09:06:18.667 Info: 8003-cq: at 4:11: Selected 1 fragment to filter
>
> That looks exactly as expected (aside from the duplication of b=c, anyway).
> If we increment the date to the 13th or the 31st, both fragments match. If we
> try to use a QName that doesn't have a range index, we lose the "Compar
[MarkLogic Dev General] Range Index in Where Clause
I'm wondering if ML uses a range index in the where portion of a FLOWR expression where applicable. I could only find examples of 'order by'. Thanks. David___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
[MarkLogic Dev General] XQuery 3.0
I was just curious what the roadmap is for the following XQuery 3.0 features: 1. group by clause in FLWOR Expressions 2. allowing empty in for clause 3. Dynamic function calls 4. count clause in FLWOR Expressions Thanks, David___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Scala and XCC
Some great thoughts David. XQuery certainly blurs the lines between data and presentation in a way that is very intriguing to me. I look forward to exploring the fuzziness more in the future :) David On Oct 19, 2012, at 10:04 AM, David Lee wrote: >>> Thanks again to everyone for their suggestions. Very enlightening. > > One more thing I would like to add. > Since there are *so many ways* to do things, it becomes important to > establish guidelines, or > dare I say a "philosophy" of how to choose. > The worse of these is "because I can" and a reasonably bad one, IMHO is > "because its easy" - i.e. easy "right now" but will it be best later ? Or > even easier later once you are more familiar . > > Things to consider include (but by far not limited to) > > * Are you running a 2 or 3 tier architecture (and why) > * If 3 tier what information does your third tier need to know (and why). > E.g. does the 3rd tier need to know and query structure of results or can it > just spit them out exactly ... > * What granularity of structure does your 3rd tier need to know ? (and why) > * If 3 tier what information does your second tier have access to (and why), > i.e. can your 2nd tier reasonably produce HTML ? does it have the domain > knowledge ? or would you rather leave that to the third tier (again a good > reason to consider a 2 tier architecture as an alternative). > > * What is the bandwidth between tiers ? > * What organizations are responsible for coding each tier and how well do > they communicate ? > * How important is speed vs ease of coding ? > > So many things to consider. I suggest starting with those questions before > coming to the question of "how to do it". > > > > > > > > - > David Lee > Lead Engineer > MarkLogic Corporation > d...@marklogic.com > Phone: +1 812-482-5224 > Cell: +1 812-630-7622 > www.marklogic.com > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Scala and XCC
Thanks again to everyone for their suggestions. Very enlightening.
David
On Oct 18, 2012, at 10:54 PM, Erik Hennum wrote:
> Hi, David:
>
> As Damon points out, two-tier XQuery web apps are extraordinarily productive.
>
> If you want to do a 3-tier web app with JVM in the middle tier, you have lots
> of choices even
> in pure Java for going from the JSON or XML model to the webapp controller.
>
> To summarize some of the most typical:
>
> * If you don't have any pre-existing XML, you can use JAXB to roundtrip POJO
> tree structures to
>and from XML. JAXB is simple when the data structure design is sourced in
> Java instead of XML.
>
> * If you have pre-existing XML, you can use a tree API such as JDOM or XOM
> or a streaming
> API such as XStream.
>
> * If you have pre-existing JSON, you can use GSON or Jackson as a tree or
> streaming API.
>
> As you note, some JVM languages like Groovy and Scala have special facilities
> for structured
> markup. If those don't meet your requirements, the JVM language can still
> use the Java libraries
> above.
>
>
> Erik Hennum
>
>
> From: general-boun...@developer.marklogic.com
> [general-boun...@developer.marklogic.com] On Behalf Of Damon Feldman
> [damon.feld...@marklogic.com]
> Sent: Thursday, October 18, 2012 7:08 PM
> To: MarkLogic Developer Discussion
> Subject: Re: [MarkLogic Dev General] Scala and XCC
>
> David,
>
> I think that's where you're going to develop new ideas. I would do it in a
> view layer inside MarkLogic - renderEmployees.xqy or something like this:
>
> let $employees :=
>
>
>
>Bob
>Smith
>
>1234 Oak St.
>San Diego
>CA
>
>
>
>
>
>Sally
>Smith
>
>1234 Elm St.
>Los Angeles
>CA
>
>
>
>
> for $e in $employees/employee
> return xdmp:document-insert(concat("/employee/",
> $e/name/@id/string<mailto:$e/name/@id/string>()), $e)
> ; (: transaction end. start new tx :)
>
>
>
> {
> for $e in /employee
> return
>{concat($e/name/first, " ",
> $e/name/last)}{$e//state/text()}
> }
>
>
>
>
> Or you could send XML right to a smart table widget in the browser, or send
> JSON via the json libraries. But you may find a better way to use Scala too.
>
> Yours,
> Damon
>
>
> From: general-boun...@developer.marklogic.com
> [general-boun...@developer.marklogic.com] On Behalf Of David Sargeant
> [da...@dsargeant.com]
> Sent: Thursday, October 18, 2012 9:40 PM
> To: MarkLogic Developer Discussion
> Subject: Re: [MarkLogic Dev General] Scala and XCC
>
> How would you normally process a nested xml structure after retrieving it?
> Or would you transform in MarkLogic first? For example, let's say I had the
> following stored in the db and wanted to display it on a web page. I'm using
> an MVC architecture, so the HTML for my view is already defined, and I just
> want to iterate over the nested structure.
>
>
>
>
>Bob
>Smith
>
>1234 Oak St.
>San Diego
>CA
>
>
>
>
>
>Sally
>Smith
>
>1234 Elm St.
>Los Angeles
>CA
>
>
>
>
>
> On Oct 18, 2012, at 7:40 PM, Damon Feldman
> mailto:damon.feld...@marklogic.com>> wrote:
>
> David,
>
>
>
> I like how you are thinking - why not have good XML support all the way
> through the stack? You should be able to do more in Scala than in Java, since
> Java's XML handling is so painful.
>
>
>
> I think your division of work between XQuery/XSLT in the MarkLogic Server and
> Scala processing may depend on speed and data volumes. If the operation is
> going to turn a lot of data into a little data, or do heavy processing, do it
> in MarkLogic.
>
> E.g. if you want the SSNs of all the People named Bob you could have Scala
> invoke
>
> /Person
>
> in MarkLogic and filter out everyone other than Bob, then throw away all data
> but the SSN. Or you could invoke
>
> /Person[name="Bob"]
>
> in MarkLogic and use Scala to pull the SSN from the resulting full
Re: [MarkLogic Dev General] Scala and XCC
How would you normally process a nested xml structure after retrieving it? Or would you transform in MarkLogic first? For example, let's say I had the following stored in the db and wanted to display it on a web page. I'm using an MVC architecture, so the HTML for my view is already defined, and I just want to iterate over the nested structure. Bob Smith 1234 Oak St. San Diego CA Sally Smith 1234 Elm St. Los Angeles CA On Oct 18, 2012, at 7:40 PM, Damon Feldman wrote: > David, > > I like how you are thinking - why not have good XML support all the way > through the stack? You should be able to do more in Scala than in Java, since > Java's XML handling is so painful. > > I think your division of work between XQuery/XSLT in the MarkLogic Server and > Scala processing may depend on speed and data volumes. If the operation is > going to turn a lot of data into a little data, or do heavy processing, do it > in MarkLogic. > > E.g. if you want the SSNs of all the People named Bob you could have Scala > invoke > > /Person > > in MarkLogic and filter out everyone other than Bob, then throw away all data > but the SSN. Or you could invoke > > /Person[name="Bob"] > > in MarkLogic and use Scala to pull the SSN from the resulting full XML > doc(s). More interesting is only moving the SSN to minimize serialization, > parsing, GC and network traffic. Do this in MarkLogic: > > /Person[name="Bob"]//ssn.text() > > and just use the resulting string in Scala. If you do return XML, I suspect > Scala's XML parsing (scala.xml.pull.Source ?) will parse it to the Scala mode > most efficiently. > > Damon > > -- > Damon Feldman, > Sr. Principal Consultant, MarkLogic > > > From: general-boun...@developer.marklogic.com > [general-boun...@developer.marklogic.com] On Behalf Of David Lee > [david@marklogic.com] > Sent: Thursday, October 18, 2012 4:09 PM > To: MarkLogic Developer Discussion > Subject: Re: [MarkLogic Dev General] Scala and XCC > > First off, are there other things you are doing that need Scala ? > I think you will find the XQuery support native to MarkLogic to be vastly > better then the XML support in Scala, > easier to use and more efficient. > > But supposing you do need or want scala. Our latest MarkLogic 6.0 has a new > Java API > I belive Scala runs in the Java JVM and can call Java directly ? > If so this may be what you are looking for > http://developer.marklogic.com/products/java > > Also you can call MarkLogic using a REST (plain old HTTP) request and get > back XML which you can then (hopefully) get Scala to parse and turn into a > Scala XML element. > > > > - > David Lee > Lead Engineer > MarkLogic Corporation > d...@marklogic.com > Phone: +1 812-482-5224 > Cell: +1 812-630-7622 > www.marklogic.com > > > > -Original Message- > From: general-boun...@developer.marklogic.com > [mailto:general-boun...@developer.marklogic.com] On Behalf Of David Sargeant > Sent: Thursday, October 18, 2012 4:04 PM > To: MarkLogic Developer Discussion > Subject: Re: [MarkLogic Dev General] Scala and XCC > > In Scala I can do the following (examples from a REPL session): > > scala> val david = > Daviddeveloper > david: scala.xml.Elem = > Daviddeveloper > > scala> (david \ "name").text > res3: String = David > > I'm wondering how to query MarkLogic using XCC and transform the results into > a scala.xml.Elem object so I can easily query the xml or map over it. See > the following slides for some more things you can do with xml in Scala - > http://www.dzone.com/links/r/xml_processing_in_scala.html. > > > David > > On Oct 18, 2012, at 3:28 PM, David Lee wrote: > > > To help answer you, could you provide details ? > > What are your goals for "builtin xml processing in scala" and how does this > > relate to your intended use of MarkLogic ? > > > > > > ----- > > David Lee > > Lead Engineer > > MarkLogic Corporation > > d...@marklogic.com > > Phone: +1 812-482-5224 > > Cell: +1 812-630-7622 > > www.marklogic.com > > > > > > > > -Original Message- >
Re: [MarkLogic Dev General] Scala and XCC
Thanks to everyone for the replies. BTW, MarkLogic and XQuery about the coolest thing I have seen in a long time. So glad to have found an elegant alternative to SQL and the mess that is tabular, rigid, relational data. David On Oct 18, 2012, at 4:38 PM, Clark Richey wrote: > If you are using MarkLogic 6 try accessing the new java client API from > scala. That should work fine and the new client API gives you much higher > level, easier to use access to MarkLogic that should easily integrate into > your application. > > Clark D. Richey, Jr. > Technical Director > MarkLogic Corporation > clark.ric...@marklogic.com > Cell: +1 240 252 7507 > www.marklogic.com > > Sent from my iPhone > > On Oct 18, 2012, at 15:16, "David Sargeant" wrote: > >> I'm farily new to the world of Java/Scala and MarkLogic, so forgive me if >> the question seems trivial. I'm wondering what the best way is to use >> MarkLogic and Scala so that I can take advantage of the built-in xml >> processing capabilities of Scala? Thanks. >> >> >> David >> ___ >> General mailing list >> General@developer.marklogic.com >> http://developer.marklogic.com/mailman/listinfo/general > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Scala and XCC
In Scala I can do the following (examples from a REPL session): scala> val david = Daviddeveloper david: scala.xml.Elem = Daviddeveloper scala> (david \ "name").text res3: String = David I'm wondering how to query MarkLogic using XCC and transform the results into a scala.xml.Elem object so I can easily query the xml or map over it. See the following slides for some more things you can do with xml in Scala - http://www.dzone.com/links/r/xml_processing_in_scala.html. David On Oct 18, 2012, at 3:28 PM, David Lee wrote: > To help answer you, could you provide details ? > What are your goals for "builtin xml processing in scala" and how does this > relate to your intended use of MarkLogic ? > > > - > David Lee > Lead Engineer > MarkLogic Corporation > d...@marklogic.com > Phone: +1 812-482-5224 > Cell: +1 812-630-7622 > www.marklogic.com > > > > -Original Message- > From: general-boun...@developer.marklogic.com > [mailto:general-boun...@developer.marklogic.com] On Behalf Of David Sargeant > Sent: Thursday, October 18, 2012 3:19 PM > To: General MarkLogic Developer Discussion > Subject: [MarkLogic Dev General] Scala and XCC > > I'm farily new to the world of Java/Scala and MarkLogic, so forgive me if the > question seems trivial. I'm wondering what the best way is to use MarkLogic > and Scala so that I can take advantage of the built-in xml processing > capabilities of Scala? Thanks. > > > David > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general > ___ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
[MarkLogic Dev General] Scala and XCC
I'm farily new to the world of Java/Scala and MarkLogic, so forgive me if the question seems trivial. I'm wondering what the best way is to use MarkLogic and Scala so that I can take advantage of the built-in xml processing capabilities of Scala? Thanks. David ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
