Re: [MarkLogic Dev General] using xdmp:eval in REST Service
HI Pragya. I did not supply you with the exact issue as I did not try to install and run all 900 lines of code. INstead, I looked at items that seemed suspect for you to log and verify what is happening with the data between the rewriter and the invoker. I am still not convinced that you have the XML you expect at the very early stages. IN fact, I'm not convinced that you ever retrieve it from the start and even once you do, I still believe you need to use xdmp:unquote. That's why I suggested you look in those two sets of code (qualifying this suggestion by experience and not having actually found your issue exactly.). But now that there is a log related to some code, lets walk through this: [as provided by you today] I tried logging in invoker: let $_populate-map := for $each-param in $urlParams let $param-value := xdmp:get-request-field($each-param) let $_ := xdmp:log(fn:concat($param-value,$param-value)) ... logs: 2014-10-17 11:16:29.008 Info: rest-ingestion: module-invoker.xqy?format=xmlmoduleName=add-documentrequestDirectory=addapiVersion=v1 2014-10-17 11:16:29.013 Info: rest-ingestion: $param-value*xml* 2014-10-17 11:16:29.013 Info: rest-ingestion: $param-valueadd-document 2014-10-17 11:16:29.013 Info: rest-ingestion: $param-valueadd 2014-10-17 11:16:29.013 Info: rest-ingestion: $param-valuev1 2014-10-17 11:16:29.013 Info: rest-ingestion: $param-value39932186-9cab-44e9-8f4f-7ebf45dabf8f 2014-10-17 11:16:29.309 Info: rest-ingestion: Ingestion Service End Point add-document.xqy 39932186-9cab-44e9-8f4f-7ebf45dabf8f 2014-10-17 11:16:29.309 Info: rest-ingestion: Path/processing/39932186-9cab-44e9-8f4f-7ebf45dabf8f/validDocs/ 2014-10-17 11:16:29.309 Info: rest-ingestion: uri/docs/BONY_4076_REPO-GMR-2000.xml So its xml at this stage. 1) Lets look at the log: The first thing I note is that you state that you do have XML and you made bold a line above. I believe you are being mislead by showing $param-value instead of the actual name of the parameter. From what I see, the *XML *that you made bold is nothing more than the value of the FORMAT parameter from your query string and not at all related to your XML content you are looking for. You can verify this with a cleaner log line and see what request field name is matching the value. 2) OK.. The where IS the content? Lets go back to an earlier message: You POST the data. So, now we have to think about how MarkLogic handles the body of a POST (not something for you to control in your code, but by the REQUEST header: - IF and only IF the request header* content-type: application/x-www-form-urlencoded* was present in the request, then the content of the request body is available via xdmp:get-request field() - Otherwise: The content is still stuck in the body and needs to be retrieved via xdmp:get-request-body() So, since you use POST, you also have to go back and verify if you are forcing the content-type: application/x-www-form-urlencoded header. Only then will you know where to troubleshoot further (because then you will know where your content is actually available) --- I hope this helps. Kind Regards, David Ennis Kind Regards, David Ennis David Ennis *Content Engineer* [image: HintTech] http://www.hinttech.com/ Mastering the value of content creative | technology | content Delftechpark 37i 2628 XJ Delft The Netherlands T: +31 88 268 25 00 M: +31 63 091 72 80 [image: http://www.hinttech.com] http://www.hinttech.com https://twitter.com/HintTech http://www.facebook.com/HintTech http://www.linkedin.com/company/HintTech On 17 October 2014 07:50, Kapoor, Pragya pkapo...@innodata.com wrote: Hi David, I tried logging in invoker: let $_populate-map := for $each-param in $urlParams let $param-value := xdmp:get-request-field($each-param) let $_ := xdmp:log(fn:concat($param-value,$param-value)) ... logs: 2014-10-17 11:16:29.008 Info: rest-ingestion: module-invoker.xqy?format=xmlmoduleName=add-documentrequestDirectory=addapiVersion=v1 2014-10-17 11:16:29.013 Info: rest-ingestion: $param-value*xml* 2014-10-17 11:16:29.013 Info: rest-ingestion: $param-valueadd-document 2014-10-17 11:16:29.013 Info: rest-ingestion: $param-valueadd 2014-10-17 11:16:29.013 Info: rest-ingestion: $param-valuev1 2014-10-17 11:16:29.013 Info: rest-ingestion: $param-value39932186-9cab-44e9-8f4f-7ebf45dabf8f 2014-10-17 11:16:29.309 Info: rest-ingestion: Ingestion Service End Point add-document.xqy 39932186-9cab-44e9-8f4f-7ebf45dabf8f 2014-10-17 11:16:29.309 Info: rest-ingestion: Path/processing/39932186-9cab-44e9-8f4f-7ebf45dabf8f/validDocs/ 2014-10-17 11:16:29.309 Info: rest-ingestion: uri/docs/BONY_4076_REPO-GMR-2000.xml So its xml at this stage. At rewriter: case element(POST) return
[MarkLogic Dev General] Element Range Query with DateTime and Durations
Hi All, Is possible to do a dateTime range query based on the elements value plus or minus a xs:dayTimeDuration. For example I want to find all articles that are 90 days passed the published date. The documentation (http://docs.marklogic.com/cts:element-range-query) gives a few examples of date queriers but they all have a supplied dateTime. Thanks Drew ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Is there a way to extract worksheet metadata from an Excel 97/2003?
If it's feasible to do your metadata extraction upstream of MarkLogic (i.e.,
before insertion) you might take a look at Apache Tika. It's designed for this
sort of thing.
You could also setup it up in a simple web service callable from MarkLogic.
POST the spreadsheet to it and have it return the metadata in whatever form you
like.
---
Ron Hitchens {r...@overstory.co.uk} +44 7879 358212
On Oct 17, 2014, at 3:35 PM, Gary Russo garyru...@hotmail.com wrote:
Hello Dennis,
Thanks for the info.
Yes, I tried xdmp:excel-convert() but this does not get the worksheet
metadata either.
The metadata that I need to retrieve from the older excel format is the
“Named Fields”.
Users create them using the Excel “Named Box” feature as shown here. =
http://spreadsheets.about.com/od/exceltips/qt/81225namebox.htm
It looks like my only option is to use the Apache POI Java API to extract the
named fields or use it to convert xls-to-xlsx on-the-fly.
=https://poi.apache.org/apidocs
I know there’s a hidden way to use MarkLogic’s underlying JVM.
It would be great if I could use it to call the Apache POI code.
But that’s a question for another day.
Thanks again,
Gary Russo
Gary Russo
Enterprise NoSQL Developer
http://garyrusso.wordpress.com
http://twitter.com/garyprusso
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of David Ennis
Sent: Thursday, October 16, 2014 5:02 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Is there a way to extract worksheet
metadata from an Excel 97/2003?
HI.
I believe that with the conversion licence, you can do what you want with:
xdmp:excel-convert
Barring that, you could always run openoffice as a headless server for
conversion purposes.
Kind Regards,
David Ennis
Kind Regards,
David Ennis
David Ennis
Content Engineer
Mastering the value of content
creative | technology | content
Delftechpark 37i
2628 XJ Delft
The Netherlands
T: +31 88 268 25 00
M: +31 63 091 72 80
On 16 October 2014 20:00, Gary Russo garyru...@hotmail.com wrote:
I need to extract worksheet metadata called “defined name” from Excel 97/2003
formatted spreadsheets.
The ISYS xdmp:document-filter() API is limiting because it only extracts the
text.
It does not extract any worksheet metadata.
Does anyone know of a workaround for this?
My only thought is to upload the “Excel 97/2003” xls file and then convert it
on the server to an “Excel 2010” xlsx format.
Once it’s in an Excel 2010 format, I can easily extract the “defined name”
metadata.
This is what it looks like in “Excel 2010” files.
definedNames
definedName name=LastYrRevenue!$B$6:$B$15/definedName
definedName name=ThisYrRevenue!$C$6:$C$15/definedName
definedName name=VarianceRevenue!$D$6:$D$15/definedName
/definedNames
Thanks,
Gary Russo
Gary Russo
Enterprise NoSQL Developer
Phone: 212-404-8639
Skype: garyprusso
http://garyrusso.wordpress.com
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Element Range Query with DateTime and Durations
Hi John and Dave, In testing the provided example worked well: xquery version 1.0-ml; declare namespace html = http://www.w3.org/1999/xhtml;; let $results := cts:search( fn:doc(), cts:element-range-query(xs:QName(publishedDate), =, fn:current-date() - xs:dayTimeDuration(P90D)) )[1 to 10] let $onSaleDates := ( xs:date(2014-07-18), (: July 18 published date :) xs:date(2014-07-19), (: July 19 published date :) xs:date(2014-07-20) (: July 20 published date :) ) let $testCases := for $onSaleDate in $onSaleDates return ( $onSaleDate, ($onSaleDate = fn:current-date() - xs:dayTimeDuration(P90D)) ) return $testCases I am trying to solve a larger problem of filtering over larger datasets (5M+ documents) where the durations are variables that come from one set of documents (business rules) and filter search of other documents (articles). Basically it is to filter out content with various embargo durations. The business rulesonly state duration after published date not the actual dates the embargo ends. Would you say there is anything to watch out with from a performance standpoint? I have range indexes set up on all the fields that require calculations. Thanks Drew From: Dave Cassel dave.cas...@marklogic.commailto:dave.cas...@marklogic.com Date: Friday, October 17, 2014 12:09 PM To: Andrew Wanczowski andrew_wanczow...@condenast.commailto:andrew_wanczow...@condenast.com Subject: Re: [MarkLogic Dev General] Element Range Query with DateTime and Durations Drew, I wasn't sure how familiar you are with durations, so in case John's answer didn't give you what you need -- cts:element-range-query( xs:QName(date), , fn:current-date() - xs:dayTimeDuration(P90D) ) -- Dave Cassel Developer Community Manager MarkLogic Corporationhttp://www.marklogic.com/ Cell: +1-484-798-8720 From: Wanczowski, Andrew andrew_wanczow...@condenast.commailto:andrew_wanczow...@condenast.com Reply-To: MarkLogic Developer Discussion general@developer.marklogic.commailto:general@developer.marklogic.com Date: Friday, October 17, 2014 at 6:42 AM To: general@developer.marklogic.commailto:general@developer.marklogic.com general@developer.marklogic.commailto:general@developer.marklogic.com Subject: Re: [MarkLogic Dev General] Element Range Query with DateTime and Durations Thanks John. I'll give that a shot. On 10/17/14 9:35 AM, John Snelson john.snel...@marklogic.commailto:john.snel...@marklogic.com wrote: Work out a dateTime 90 days before the current dateTime, and query for articles with a published dateTime before that dateTime. John On 17/10/14 14:26, Wanczowski, Andrew wrote: Hi All, Is possible to do a dateTime range query based on the elements value plus or minus a xs:dayTimeDuration. For example I want to find all articles that are 90 days passed the published date. The documentation (http://docs.marklogic.com/cts:element-range-query) gives a few examples of date queriers but they all have a supplied dateTime. Thanks Drew -- John Snelson, Lead Engineerhttp://twitter.com/jpcs MarkLogic Corporation http://www.marklogic.com ___ General mailing list General@developer.marklogic.commailto:General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.commailto:General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] word query
The word Query problem was on the Marklogic version 7.0-2.1. It works fine on 7.0-3. Girish On Thu, Oct 16, 2014 at 9:45 AM, Danny Sokolsky danny.sokol...@marklogic.com wrote: Have you tried this on 7.0-4? There are some bugs fixed in this area there—it is worth a try. What I recommend you do is create a simple test case using a simple cts:search with a cts:query that shows the issue, then post that here with the exact config info for it. Otherwise we are just guessing. The details are very important to understanding what is going on. -Danny *From:* general-boun...@developer.marklogic.com [mailto: general-boun...@developer.marklogic.com] *On Behalf Of *Girish Kulkarni *Sent:* Thursday, October 16, 2014 9:39 AM *To:* MarkLogic Developer Discussion *Subject:* Re: [MarkLogic Dev General] word query i am using word-constraint-query in my structured query and i see that when i exclude the enrichedDateTime , it does work in the sense i dont see any results when i search for the timestamp search i was seeing earlier. Now my only problem is i don't see other fields being searched. Well to be more specific i see that the results returned show total = 20 but i don't see anything being returned . As soon as i add the top level field fix in the inclusion list i start seeing the 20 results. Girish On Wed, Oct 15, 2014 at 4:09 PM, Danny Sokolsky danny.sokol...@marklogic.com wrote: Depending on what those structured queries are, they might not be word queries. Excluding an element in the word query field does not mean you cannot query it, it just means that a cts:word-query will not see it. For example, you can still see it in an element-word-query. You say your reindexing started; did it complete? -Danny *From:* general-boun...@developer.marklogic.com [mailto: general-boun...@developer.marklogic.com] *On Behalf Of *Girish Kulkarni *Sent:* Wednesday, October 15, 2014 3:59 PM *To:* MarkLogic Developer Discussion *Subject:* Re: [MarkLogic Dev General] word query As soon as i make changes to the word query , my database starts re-indexing. We use marklogic 7.0-3 and we are using REST based structured queries. The query passes in a bunch of constraints and options , however i am just testing before and after scenario's based upon changes to the word query. Girish On Wed, Oct 15, 2014 at 3:55 PM, Danny Sokolsky danny.sokol...@marklogic.com wrote: Did you reindex your database after changing the word query field? Exactly what query are you running? What version of MarkLogic are you using (xdmp:version() ). -Danny *From:* general-boun...@developer.marklogic.com [mailto: general-boun...@developer.marklogic.com] *On Behalf Of *Girish Kulkarni *Sent:* Wednesday, October 15, 2014 3:48 PM *To:* MarkLogic Developer Discussion *Subject:* [MarkLogic Dev General] word query I had some fileds in my xml document like enrichedDateTime which i didnt want to index and search upon. When i added this in the word query exclusion list for some reason my search result isn't returning back this document at all even when i searched for other field like content. However when i added the root field name fix to my inclusion list i do see the document back again. I had already set the include root flag to true but seems like for some reason i am un-able to search for other fields in the document. Any ideas why this could be happening ? fix content some content goes here /content enrichedDateTime2014-09-30T16:32:27.424443-07:00/enrichedDateTime /fix Girish Kulkarni ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general -- Girish Kulkarni ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general -- Girish Kulkarni ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general -- Girish Kulkarni ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
