Hi Erik, Did you figure out how to extend the grammar?
Regards, Oleksii Segeda IT Analyst Information and Technology Solutions -----Original Message----- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Oleksii Segeda Sent: Monday, January 30, 2017 3:09 PM To: MarkLogic Developer Discussion <general@developer.marklogic.com> Subject: Re: [MarkLogic Dev General] Custom search grammar Hi Erik, Yes, that's is desired behavior. Ideally, I would like to avoid custom constraints, simply because search grammar looks cleaner in the search box. In addition, some of our users are already familiar with simple search operators like AND, OR, so BOOST won't look like an alien to them. I guess a postprocessing can be used as you suggested, however I'm interested in custom search grammar, because I may need to extend it more in the future. Thank you, Oleksii Segeda IT Analyst Information and Technology Solutions -----Original Message----- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Erik Hennum Sent: Monday, January 30, 2017 2:42 PM To: MarkLogic Developer Discussion <general@developer.marklogic.com> Subject: Re: [MarkLogic Dev General] Custom search grammar Hi, Oleksii: Thanks for providing more detail. Just to confirm, is it clear that, in a boost query, the right-hand term is optional? Documents with only the left-hand term will still appear in the results though with less relevance than documents that have both terms. By contrast, AND-related terms are both required and both contribute to relevance. Anyway, to increase weight, one approach would be to define a tag for a quoted phrase and pass the phrase to a Search API custom constraint or to cts:parse() with a binding to a query generator function: http://docs.marklogic.com/guide/search-dev/cts_query#id_13456 The custom code could then tokenize the phrase and combine the terms with a boost-query or and-query, adding appropriate weight. Another approach would be to do postprocessing of the query tree returned by cts:parse() or search:parse() to replace the default boost-query or and-query with a query that has more weight. In either approach, you would then search on the query. I mention cts:parse() because it parses query text more quickly than search:parse() Hoping that helps, Erik Hennum ________________________________________ From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Oleksii Segeda [oseg...@worldbankgroup.org] Sent: Monday, January 30, 2017 10:55 AM To: general@developer.marklogic.com Subject: Re: [MarkLogic Dev General] Custom search grammar Hi Erik, I'm trying to boost some parts of search query. For example, if user types `trade BOOST water`, I want documents with the word "water" to be higher in the results. cts:boost-query seems to be a perfect fit, but the default BOOST doesn't let you specify weights. My ultimate goal is to convert `trade BOOST water` to something like this: cts:boost-query(cts:word-query("trade"), cts:word-query("water", (), 10.0) ) Regards, Oleksii Segeda IT Analyst Information and Technology Solutions -----Original Message----- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of general-requ...@developer.marklogic.com Sent: Monday, January 30, 2017 1:08 PM To: general@developer.marklogic.com Subject: General Digest, Vol 151, Issue 42 Send General mailing list submissions to general@developer.marklogic.com To subscribe or unsubscribe via the World Wide Web, visit http://developer.marklogic.com/mailman/listinfo/general or, via email, send a message with subject or body 'help' to general-requ...@developer.marklogic.com You can reach the person managing the list at general-ow...@developer.marklogic.com When replying, please edit your Subject line so it is more specific than "Re: Contents of General digest..." Today's Topics: 1. Custom search grammar (Oleksii Segeda) 2. Re: Custom search grammar (Erik Hennum) ---------------------------------------------------------------------- Message: 1 Date: Mon, 30 Jan 2017 16:51:26 +0000 From: Oleksii Segeda <oseg...@worldbankgroup.org> Subject: [MarkLogic Dev General] Custom search grammar To: "general@developer.marklogic.com" <general@developer.marklogic.com> Message-ID: <bn1pr0101mb0769b9cdcd5e7697ace8381bcb...@bn1pr0101mb0769.prod.exchangelabs.com> Content-Type: text/plain; charset="us-ascii" Hi there, I'm trying to declare a custom search grammar. I declared a custom function via search options, which supposed to parse "BOOST" keyword: <joiner strength="2" apply="custom-boost" ns="http://worldbankgroup.org/search/grammar"; at="/lib/grammar-boost.xqy" tokenize="word">BOOST</joiner> I declared this function and just copied existing implementation from impl:joiner-boost function in /MarkLogic/appservices/search/search-impl.xqy : declare function grammar:custom-boost($ps as map:map, $left as element()?, $opts as element()?) as schema-element(cts:query) { let $symbol := impl:symbol-lookup($ps) let $_ := tdop:advance($ps) let $expr1 := tdop:expression($ps, $symbol/@strength) return if (empty($left)) then ($left, impl:msg($ps, <cts:annotation warning="SEARCH-IGNOREDQTEXT:[{string($symbol)} {$expr1}]: expected two arguments"/>)) else element { xs:QName($symbol/@element) } { attribute qtextjoin {concat($symbol/string())}, attribute strength {$symbol/@strength}, attribute qtextgroup { impl:opts($ps)/opt:grammar/opt:starter[@apply eq "grouping"]/(string(), @delimiter/string()) }, for $opt in $symbol/@options/tokenize(normalize-space(.)<mailto:$symbol/@options/tokenize(normalize-space(.)>, "\s") return <cts:option>{$opt}</cts:option>, element cts:matching-query { attribute qtextref { "schema-element(cts:query)" }, $left }, element cts:boosting-query { attribute qtextref { "schema-element(cts:query)" }, $expr1 } } }; Unfortunately this doesn't work, because for some reason impl:symbol-lookup returns an empty sequence. Any ideas what went wrong here? Oleksii Segeda IT Analyst Information and Technology Solutions [http://siteresources.worldbank.org/NEWS/Images/spacer.png] [http://siteresources.worldbank.org/NEWS/Images/WBG_Information_and_Technology_Solutions.png] -------------- next part -------------- An HTML attachment was scrubbed... URL: http://developer.marklogic.com/pipermail/general/attachments/20170130/1958bd77/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.png Type: image/png Size: 6577 bytes Desc: image003.png Url : http://developer.marklogic.com/pipermail/general/attachments/20170130/1958bd77/attachment-0002.png -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.png Type: image/png Size: 170 bytes Desc: image004.png Url : http://developer.marklogic.com/pipermail/general/attachments/20170130/1958bd77/attachment-0003.png ------------------------------ Message: 2 Date: Mon, 30 Jan 2017 18:07:41 +0000 From: Erik Hennum <erik.hen...@marklogic.com> Subject: Re: [MarkLogic Dev General] Custom search grammar To: MarkLogic Developer Discussion <general@developer.marklogic.com> Message-ID: <dfdf2fd50bf5aa42adaf93ff2e3ca1850bd7d...@exchg10-be01.marklogic.com> Content-Type: text/plain; charset="windows-1252" Hi, Oleksii: Can you explain what you are trying to accomplish? There may be better ways of doing the same thing than creating a custom grammar, which is really a tool of last resort. For instance, a custom constraint can map a term to a custom query. For other cases, it's often useful to do postprocessing on the generated query. If a custom grammar really is unavoidable, in many cases a special-purpose third-party parsing tool may provide a faster and more flexible alternative to the limited custom grammar in the Search API. For instance, the Jison.js and Peg.js parsers work with server-side JavaScript. (A nearly.js parser is also available, though I've heard no reports about it yet.) Hoping that helps, Erik Hennum ________________________________ From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Oleksii Segeda [oseg...@worldbankgroup.org] Sent: Monday, January 30, 2017 8:51 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Custom search grammar Hi there, I?m trying to declare a custom search grammar. I declared a custom function via search options, which supposed to parse ?BOOST? keyword: <joiner strength="2" apply="custom-boost" ns="http://worldbankgroup.org/search/grammar"; at="/lib/grammar-boost.xqy" tokenize="word">BOOST</joiner> I declared this function and just copied existing implementation from impl:joiner-boost function in /MarkLogic/appservices/search/search-impl.xqy : declare function grammar:custom-boost($ps as map:map, $left as element()?, $opts as element()?) as schema-element(cts:query) { let $symbol := impl:symbol-lookup($ps) let $_ := tdop:advance($ps) let $expr1 := tdop:expression($ps, $symbol/@strength) return if (empty($left)) then ($left, impl:msg($ps, <cts:annotation warning="SEARCH-IGNOREDQTEXT:[{string($symbol)} {$expr1}]: expected two arguments"/>)) else element { xs:QName($symbol/@element) } { attribute qtextjoin {concat($symbol/string())}, attribute strength {$symbol/@strength}, attribute qtextgroup { impl:opts($ps)/opt:grammar/opt:starter[@apply eq "grouping"]/(string(), @delimiter/string()) }, for $opt in $symbol/@options/tokenize(normalize-space(.)<mailto:$symbol/@options/tokenize(normalize-space(.)>, "\s") return <cts:option>{$opt}</cts:option>, element cts:matching-query { attribute qtextref { "schema-element(cts:query)" }, $left }, element cts:boosting-query { attribute qtextref { "schema-element(cts:query)" }, $expr1 } } }; Unfortunately this doesn?t work, because for some reason impl:symbol-lookup returns an empty sequence. Any ideas what went wrong here? Oleksii Segeda IT Analyst Information and Technology Solutions [http://siteresources.worldbank.org/NEWS/Images/spacer.png] [http://siteresources.worldbank.org/NEWS/Images/WBG_Information_and_Technology_Solutions.png] -------------- next part -------------- An HTML attachment was scrubbed... URL: http://developer.marklogic.com/pipermail/general/attachments/20170130/33effe92/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.png Type: image/png Size: 6577 bytes Desc: image003.png Url : http://developer.marklogic.com/pipermail/general/attachments/20170130/33effe92/attachment.png -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.png Type: image/png Size: 170 bytes Desc: image004.png Url : http://developer.marklogic.com/pipermail/general/attachments/20170130/33effe92/attachment-0001.png ------------------------------ _______________________________________________ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general End of General Digest, Vol 151, Issue 42 **************************************** _______________________________________________ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
