[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/113650 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/113650 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue job feature-ft_module-2012-07-09T21-17-04.718Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/113650 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-07-09T21-17-04.718Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/113650 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/113650 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/113650 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/113650 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/113650 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Paul J. Lucas has proposed merging lp:~zorba-coders/zorba/feature-ft_module
into lp:zorba.
Requested reviews:
Paul J. Lucas (paul-lucas)
Related bugs:
Bug #944795 in Zorba: "XQDoc doesn't handle & in URLs"
https://bugs.launchpad.net/zorba/+bug/944795
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/113650
1. Added missing "lang" attribute to tokens generated from tokenize-nodes().
2. Added a test.
--
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/113650
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'src/runtime/full_text/ft_module_util.cpp'
--- src/runtime/full_text/ft_module_util.cpp 2012-06-22 03:32:00 +
+++ src/runtime/full_text/ft_module_util.cpp 2012-07-05 21:50:26 +
@@ -34,7 +34,7 @@
Item const *api_item ) {
store::Item const *const item = Unmarshaller::getInternalItem( *api_item );
tokens_->push_back(
-FTToken( utf8_s, utf8_len, token_no, sent_no, para_no, item )
+FTToken( utf8_s, utf8_len, token_no, sent_no, para_no, item, lang )
);
}
=== added file 'test/rbkt/ExpQueryResults/zorba/fulltext/ft-module-tokenize-nodes-2.xml.res'
--- test/rbkt/ExpQueryResults/zorba/fulltext/ft-module-tokenize-nodes-2.xml.res 1970-01-01 00:00:00 +
+++ test/rbkt/ExpQueryResults/zorba/fulltext/ft-module-tokenize-nodes-2.xml.res 2012-07-05 21:50:26 +
@@ -0,0 +1,1 @@
+true
=== added file 'test/rbkt/Queries/zorba/fulltext/ft-module-tokenize-nodes-2.xq'
--- test/rbkt/Queries/zorba/fulltext/ft-module-tokenize-nodes-2.xq 1970-01-01 00:00:00 +
+++ test/rbkt/Queries/zorba/fulltext/ft-module-tokenize-nodes-2.xq 2012-07-05 21:50:26 +
@@ -0,0 +1,33 @@
+import module namespace ft = "http://www.zorba-xquery.com/modules/full-text";;
+import schema namespace fts = "http://www.zorba-xquery.com/modules/full-text";;
+
+let $x :=
+
+
+ I love you more than yesterday, less than tomorrow.
+
+
+ Je t'aime plus qu'hier, moins que demain.
+
+
+
+let $includes := $x//quote
+let $excludes := ()
+
+let $tokens := ft:tokenize-nodes( $includes, $excludes )
+
+let $t1 := validate { $tokens[1] }
+let $t2 := validate { $tokens[2] }
+let $t3 := validate { $tokens[3] }
+
+let $t10 := validate { $tokens[10] }
+let $t11 := validate { $tokens[11] }
+
+return $t1/@value = "I"and $t1/@lang = "en"
+and $t2/@value = "love" and $t2/@lang = "en"
+and $t3/@value = "you" and $t2/@lang = "en"
+
+and $t10/@value = "Je" and $t10/@lang = "fr"
+and $t11/@value = "t'aime" and $t11/@lang = "fr"
+
+(: vim:set et sw=2 ts=2: :)
--
Mailing list: https://launchpad.net/~zorba-coders
Post to : zorba-coders@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zorba-coders
More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/109457 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/109457 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue job feature-ft_module-2012-06-09T00-08-09.806Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/109457 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-06-09T00-08-09.806Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/109457 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/109457 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/109457 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/109457 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/109457 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Paul J. Lucas has proposed merging lp:~zorba-coders/zorba/feature-ft_module
into lp:zorba.
Requested reviews:
Paul J. Lucas (paul-lucas)
Matthias Brantner (matthias-brantner)
Related bugs:
Bug #944795 in Zorba: "XQDoc doesn't handle & in URLs"
https://bugs.launchpad.net/zorba/+bug/944795
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/109457
1. Fixed return type of ft:thesaurus-lookup().
2. Added a test to ensure that a look-up of a non-existant word works.
--
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/109457
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'modules/com/zorba-xquery/www/modules/full-text.xq'
--- modules/com/zorba-xquery/www/modules/full-text.xq 2012-05-17 22:44:24 +
+++ modules/com/zorba-xquery/www/modules/full-text.xq 2012-06-08 23:39:24 +
@@ -636,7 +636,7 @@
: The phrase's
: http://www.w3.org/TR/xmlschema-2/#language";>language
: is assumed to be the one returned by ft:current-lang().
- : @return the original and related phrases.
+ : @return the related phrases.
: @error err:FTST0009 if ft:current-lang() is not supported.
: @error zerr:ZXQP8401 if the thesaurus data file's version is not supported
: by the currently running version of Zorba.
@@ -646,7 +646,7 @@
: @example test/rbkt/Queries/zorba/fulltext/ft-module-thesaurus-lookup-1.xq
:)
declare function ft:thesaurus-lookup( $phrase as xs:string )
- as xs:string+ external;
+ as xs:string* external;
(:~
: Looks-up the given phrase in the thesaurus specified by the given URI.
@@ -656,7 +656,7 @@
: @param $lang The
: http://www.w3.org/TR/xmlschema-2/#language";>language
: of $phrase.
- : @return the original and related phrases.
+ : @return the related phrases.
: @error err:FTST0009 if $lang is not supported.
: @error err:FTST0018 if $uri refers to a thesaurus
: that is not found in the statically known thesauri.
@@ -671,7 +671,7 @@
:)
declare function ft:thesaurus-lookup( $uri as xs:string, $phrase as xs:string,
$lang as xs:language )
- as xs:string+ external;
+ as xs:string* external;
(:~
: Looks-up the given phrase in a thesaurus.
@@ -681,7 +681,7 @@
: The phrase's
: http://www.w3.org/TR/xmlschema-2/#language";>language
: is assumed to be the one the one returned by ft:current-lang().
- : @return the original and related phrases.
+ : @return the related phrases.
: @error err:FTST0009 if ft:current-lang() is unsupported.
: @error err:FTST0018 if $uri refers to a thesaurus
: that is not found in the statically known thesauri.
@@ -695,7 +695,7 @@
: @example test/rbkt/Queries/zorba/fulltext/ft-module-thesaurus-lookup-3.xq
:)
declare function ft:thesaurus-lookup( $uri as xs:string, $phrase as xs:string )
- as xs:string+ external;
+ as xs:string* external;
(:~
: Looks-up the given phrase in a thesaurus.
@@ -707,7 +707,7 @@
: of $phrase.
: @param $relationship The relationship the results are to have to
: $phrase.
- : @return the original and related phrases.
+ : @return the related phrases.
: @error err:FTST0018 if $uri refers to a thesaurus
: that is not found in the statically known thesauri.
: @error err:FTST0009 if $lang is not supported.
@@ -723,7 +723,7 @@
declare function ft:thesaurus-lookup( $uri as xs:string, $phrase as xs:string,
$lang as xs:language,
$relationship as xs:string )
- as xs:string+ external;
+ as xs:string* external;
(:~
: Looks-up the given phrase in a thesaurus.
@@ -739,7 +739,7 @@
: travers$ed.
: @param $level-most The maximum number of levels within the thesaurus to be
: traversed.
- : @return the original and related phrases.
+ : @return the related phrases.
: @error err:FOCA0003 if either $level-least or
: $level-most is either negative or too large.
: @error err:FTST0018 if $uri refers to a thesaurus
@@ -759,7 +759,7 @@
$relationship as xs:string,
$level-least as xs:integer,
$level-most as xs:integer )
- as xs:string+ external;
+ as xs:string* external;
(:~
: Tokenizes the given node and all of its descendants.
=== modified file 'src/functions/pregenerated/func_ft_module.cpp'
--- src/functions/pregenerated/func_ft_module.cpp 2012-04-26 13:25:13 +
+++ src/functions/pregenerated/func_ft_module.cpp 2012-06-08 23:39:24 +
@@ -366,7 +366,7 @@
DECL_WITH_KIND(sctx, full_text_thesaurus_lookup,
(createQName("http://www.zorba-xquery.com/modules/full-text","","thesaurus-lookup";),
GENV_TYPESYSTEM.STRING_TYPE_ONE,
-GENV_TYPESYSTEM.STRING_TYPE_PLUS),
+GENV_TYPESYSTEM.STRING_TYPE_STAR),
FunctionConsts::FULL_TEXT_THESAURUS_LOOKUP_1);
}
@@ -383,7 +383,7 @@
(createQName("http://www.zorba-xquery.com/mo
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue job feature-ft_module-2012-05-19T00-43-06.648Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-19T00-43-06.648Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve The comment is wrong: #endif /* __GNUC__ */ -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Until it's actually needed in more than one place, I don't see the point. -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Needs Information Why not put the definition of the preprocessor variable into a common place such as zorba/include/zorba/config.h.cmake? This file already contains similar definitions. -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Paul J. Lucas has proposed merging lp:~zorba-coders/zorba/feature-ft_module
into lp:zorba.
Requested reviews:
Matthias Brantner (matthias-brantner)
Paul J. Lucas (paul-lucas)
Related bugs:
Bug #944795 in Zorba: "XQDoc doesn't handle & in URLs"
https://bugs.launchpad.net/zorba/+bug/944795
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480
Fixed a couple of warnings.
--
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106480
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'src/runtime/full_text/ft_module_impl.cpp'
--- src/runtime/full_text/ft_module_impl.cpp 2012-05-17 18:16:26 +
+++ src/runtime/full_text/ft_module_impl.cpp 2012-05-18 23:24:59 +
@@ -423,7 +423,15 @@
///
-#ifdef __GNUC__
+#if defined( __GNUC__ ) && (__GNUC__ * 100 + __GNUC_MINOR__ >= 460)
+# define GCC_GREATER_EQUAL_460 1
+#endif
+
+#if defined( GCC_GREATER_EQUAL_460 ) || defined( __llvm__ )
+# define GCC_PRAGMA_DIAGNOSTIC_PUSH 1
+#endif
+
+#ifdef GCC_PRAGMA_DIAGNOSTIC_PUSH
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wbind-to-temporary-copy"
#endif /* __GNUC__ */
@@ -522,7 +530,7 @@
ZORBA_ASSERT( state->tresult_.get() );
}
-#ifdef __GNUC__
+#ifdef GCC_PRAGMA_DIAGNOSTIC_PUSH
# pragma GCC diagnostic pop
#endif /* __GNUC__ */
=== modified file 'src/util/icu_streambuf.cpp'
--- src/util/icu_streambuf.cpp 2012-05-10 04:08:30 +
+++ src/util/icu_streambuf.cpp 2012-05-18 23:24:59 +
@@ -163,8 +163,12 @@
char_type const *from = &utf8_byte;
char ebuf[ Small_External_Buf_Size ], *to = ebuf;
+#ifdef NDEBUG
+ to_external( &from, from + 1, &to, to + sizeof ebuf );
+#else
bool const ok = to_external( &from, from + 1, &to, to + sizeof ebuf );
assert( ok );
+#endif /* NDEBUG */
if ( streamsize const n = to - ebuf ) {
original()->sputn( ebuf, n );
p_.reset();
--
Mailing list: https://launchpad.net/~zorba-coders
Post to : zorba-coders@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zorba-coders
More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue job feature-ft_module-2012-05-18T03-45-08.316Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-18T03-45-08.316Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Needs review For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The attempt to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba failed. Below is the output from the failed tests. CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message): Validation queue job feature-ft_module-2012-05-17T21-57-06.963Z is finished. The final status was: 1 tests did not succeed - changes not commited. Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-17T21-57-06.963Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Paul J. Lucas has proposed merging lp:~zorba-coders/zorba/feature-ft_module
into lp:zorba.
Requested reviews:
Paul J. Lucas (paul-lucas)
Matthias Brantner (matthias-brantner)
Related bugs:
Bug #944795 in Zorba: "XQDoc doesn't handle & in URLs"
https://bugs.launchpad.net/zorba/+bug/944795
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235
Getting in another public API change for 2.5 for the full-text module since
now's the time to do it. Renamed tokenize() to tokenize-node() for 2 reasons:
1. There already exists tokenize-string() and therefore tokenize-node() is a
better name than just plain tokenize().
2. The forthcoming addition of the black & white tokenization function will
most likely be called tokenize-nodes() -- plural.
--
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/106235
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'doc/zorba/ft_tokenizer.dox'
--- doc/zorba/ft_tokenizer.dox 2012-05-16 01:01:06 +
+++ doc/zorba/ft_tokenizer.dox 2012-05-17 18:22:21 +
@@ -152,7 +152,7 @@
-A complete implementation of \c %tokenize() is non-trivial
+A complete implementation of \c %tokenize_string() is non-trivial
and therefore an example is beyond the scope of this API documentation.
However,
the things a tokenizer should take into consideration include:
=== modified file 'modules/com/zorba-xquery/www/modules/full-text.xq'
--- modules/com/zorba-xquery/www/modules/full-text.xq 2012-05-08 23:49:22 +
+++ modules/com/zorba-xquery/www/modules/full-text.xq 2012-05-17 18:22:21 +
@@ -762,7 +762,7 @@
as xs:string+ external;
(:~
- : Tokenizes the given document.
+ : Tokenizes the given node and all of its descendants.
:
: @param $node The node to tokenize.
: @param $lang The default
@@ -772,11 +772,11 @@
: @error err:FTST0009 if $lang is not supported in general.
: @example test/rbkt/Queries/zorba/fulltext/ft-module-tokenize-1.xq
:)
-declare function ft:tokenize( $node as node(), $lang as xs:language )
+declare function ft:tokenize-node( $node as node(), $lang as xs:language )
as element(ft-schema:token)* external;
(:~
- : Tokenizes the given document.
+ : Tokenizes the given node and all of its descendants.
:
: @param $node The node to tokenize.
: The document's default
@@ -789,7 +789,7 @@
: @example test/rbkt/Queries/zorba/fulltext/ft-module-tokenize-3.xq
: @example test/rbkt/Queries/zorba/fulltext/ft-module-tokenize-4.xq
:)
-declare function ft:tokenize( $node as node() )
+declare function ft:tokenize-node( $node as node() )
as element(ft-schema:token)* external;
(:~
=== modified file 'src/functions/func_ft_module_impl.cpp'
--- src/functions/func_ft_module_impl.cpp 2012-05-15 21:13:21 +
+++ src/functions/func_ft_module_impl.cpp 2012-05-17 18:22:21 +
@@ -25,14 +25,14 @@
#ifndef ZORBA_NO_FULL_TEXT
-PlanIter_t full_text_tokenize::codegen(
+PlanIter_t full_text_tokenize_node::codegen(
CompilerCB*,
static_context* sctx,
const QueryLoc& loc,
std::vector& argv,
expr& ann) const
{
- return new TokenizeIterator(sctx, loc, argv);
+ return new TokenizeNodeIterator(sctx, loc, argv);
}
@@ -90,20 +90,20 @@
false);
{
DECL_WITH_KIND(sctx,
- full_text_tokenize,
- (createQName(FT_MODULE_NS, "", "tokenize"),
+ full_text_tokenize_node,
+ (createQName(FT_MODULE_NS, "", "tokenize-node"),
GENV_TYPESYSTEM.ANY_NODE_TYPE_ONE,
tokenize_return_type),
- FunctionConsts::FULL_TEXT_TOKENIZE_1);
+ FunctionConsts::FULL_TEXT_TOKENIZE_NODE_1);
}
{
DECL_WITH_KIND(sctx,
- full_text_tokenize,
- (createQName( FT_MODULE_NS, "", "tokenize"),
+ full_text_tokenize_node,
+ (createQName( FT_MODULE_NS, "", "tokenize-node"),
GENV_TYPESYSTEM.ANY_NODE_TYPE_ONE,
GENV_TYPESYSTEM.LANGUAGE_TYPE_ONE,
tokenize_return_type),
- FunctionConsts::FULL_TEXT_TOKENIZE_2);
+ FunctionConsts::FULL_TEXT_TOKENIZE_NODE_2);
}
xqtref_t tokenizer_properties_return_type =
=== modified file 'src/functions/func_ft_module_impl.h'
--- src/functions/func_ft_module_impl.h 2012-05-09 20:40:03 +
+++ src/functions/func_ft_module_impl.h 2012-05-17 18:22:21 +
@@ -30,11 +30,11 @@
///
//full-text:tokenize
-class full_text_tokenize : public function
+class full_text_tokenize_node : public function
{
public:
- full_text_tokenize(const signature& sig, FunctionConsts::FunctionKind kind)
-:
+ full_text_tokenize_node(const signature& sig,
+ FunctionConsts::FunctionKind kind) :
functio
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105914 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105914 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue job feature-ft_module-2012-05-16T18-36-29.153Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105914 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-16T18-36-29.153Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105914 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105914 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105914 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105914 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105914 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Paul J. Lucas has proposed merging lp:~zorba-coders/zorba/feature-ft_module
into lp:zorba.
Requested reviews:
Matthias Brantner (matthias-brantner)
Paul J. Lucas (paul-lucas)
Related bugs:
Bug #944795 in Zorba: "XQDoc doesn't handle & in URLs"
https://bugs.launchpad.net/zorba/+bug/944795
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105914
Renamed Tokenizer::Numbers to Tokenizer::State now (just prior to the 2.5
release) to give it a better name for the forthcoming addition of the ability
to tokenize using include/exclude Item lists. At that time, State will most
likely be expanded to include additional state information beyond just numbers,
hence the name change.
(In the previous proposal, I had forgotten to update the documentation -- now
done.)
--
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105914
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'doc/zorba/ft_tokenizer.dox'
--- doc/zorba/ft_tokenizer.dox 2012-05-10 17:51:56 +
+++ doc/zorba/ft_tokenizer.dox 2012-05-16 01:06:11 +
@@ -38,14 +38,14 @@
typedef /* implementation-defined */ ptr;
typedef /* implementation-defined */ size_type;
- struct Numbers {
+ struct State {
typedef Tokenizer::size_type value_type;
value_type token; // Token number.
value_type sent;// Sentence number.
value_type para;// Paragraph number.
-Numbers();
+State();
};
class Callback {
@@ -72,8 +72,8 @@
virtual void properties( Properties *result ) const = 0;
virtual void destroy() const = 0;
- Numbers& numbers();
- Numbers const& numbers() const;
+ State& state();
+ State const& state() const;
void tokenize_node( Item const &node, locale::iso639_1::type lang, Callback &callback );
@@ -81,7 +81,7 @@
bool wildcards, Callback &callback, Item const *item = 0 ) = 0;
protected:
- Tokenizer( Numbers& );
+ Tokenizer( State& );
virtual ~Tokenizer();
bool find_lang_attribute( Item const&, locale::iso639_1::type *lang );
@@ -95,7 +95,7 @@
and why the destructor is \c protected,
see the \ref memory_management document.
-The \c Numbers \c struct is created by Zorba
+The \c State \c struct is created by Zorba
and passed to your constructor.
It simply keeps track of the current
token, sentence, and paragraph numbers.
@@ -202,7 +202,7 @@
Item qname;
item.getNodeName( qname );
if ( /* qname matches an XHTML block-level element's name */ )
- ++numbers().para;
+ ++state().para;
}
\endcode
@@ -262,7 +262,7 @@
class TokenizerProvider {
public:
virtual ~TokenizerProvider();
- virtual bool getTokenizer( locale::iso639_1::type lang, Tokenizer::Numbers *numbers = 0, Tokenizer::ptr* = 0 ) const = 0;
+ virtual bool getTokenizer( locale::iso639_1::type lang, Tokenizer::State *state = 0, Tokenizer::ptr* = 0 ) const = 0;
};
\endcode
@@ -278,9 +278,9 @@
-\c num
+\c state
- The \c Numbers to use.
+ The \c State to use.
If \c null,
\a t is not set.
@@ -299,13 +299,13 @@
\code
class MyTokenizerProvider : public TokenizerProvider {
public:
- getTokenizer( locale::iso639_1::type lang, Tokenizer::Numbers* = 0, Tokenizer::ptr* = 0 ) const;
+ getTokenizer( locale::iso639_1::type lang, Tokenizer::State* = 0, Tokenizer::ptr* = 0 ) const;
};
-bool MyTokenizerProvider::getTokenizer( locale::iso639_1::type lang, Tokenizer::Numbers *num, Tokenizer::ptr *t ) const {
+bool MyTokenizerProvider::getTokenizer( locale::iso639_1::type lang, Tokenizer::State *state, Tokenizer::ptr *t ) const {
switch ( lang ) {
case iso639_1::en:
- if ( num && t )
+ if ( state && t )
t->reset( new MyTokenizer );
return true;
default:
=== modified file 'include/zorba/tokenizer.h'
--- include/zorba/tokenizer.h 2012-05-03 12:31:51 +
+++ include/zorba/tokenizer.h 2012-05-16 01:06:11 +
@@ -48,9 +48,10 @@
/
/**
- * A %Numbers contains the current token, sentence, and paragraph numbers.
+ * A %State contains inter-Tokenizer state, currently the current token,
+ * sentence, and paragraph numbers.
*/
- struct Numbers {
+ struct State {
typedef Tokenizer::size_type value_type;
value_type token; ///< Token number.
@@ -60,7 +61,7 @@
/**
* Default constructor.
*/
-Numbers();
+State();
};
/
@@ -125,7 +126,7 @@
/**
* If \c true, XML processing instructions separate tokens. For example,
- * network would be 2 tokens instead of 1.
+ * network would be 2 tokens instead of 1.
*/
bool processing_instructions_separate_tokens;
@@ -162,18 +163,18 @@
virtual void destroy() const = 0;
/**
- * Gets t
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105913 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Paul J. Lucas has proposed merging lp:~zorba-coders/zorba/feature-ft_module
into lp:zorba.
Requested reviews:
Matthias Brantner (matthias-brantner)
Paul J. Lucas (paul-lucas)
Related bugs:
Bug #944795 in Zorba: "XQDoc doesn't handle & in URLs"
https://bugs.launchpad.net/zorba/+bug/944795
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105913
Renamed Tokenizer::Numbers to Tokenizer::State now (just prior to the 2.5
release) to give it a better name for the forthcoming addition of the ability
to tokenize using include/exclude Item lists. At that time, State will most
likely be expanded to include additional state information beyond just numbers,
hence the name change.
--
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105913
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'include/zorba/tokenizer.h'
--- include/zorba/tokenizer.h 2012-05-03 12:31:51 +
+++ include/zorba/tokenizer.h 2012-05-16 00:57:21 +
@@ -48,9 +48,10 @@
/
/**
- * A %Numbers contains the current token, sentence, and paragraph numbers.
+ * A %State contains inter-Tokenizer state, currently the current token,
+ * sentence, and paragraph numbers.
*/
- struct Numbers {
+ struct State {
typedef Tokenizer::size_type value_type;
value_type token; ///< Token number.
@@ -60,7 +61,7 @@
/**
* Default constructor.
*/
-Numbers();
+State();
};
/
@@ -125,7 +126,7 @@
/**
* If \c true, XML processing instructions separate tokens. For example,
- * network would be 2 tokens instead of 1.
+ * network would be 2 tokens instead of 1.
*/
bool processing_instructions_separate_tokens;
@@ -162,18 +163,18 @@
virtual void destroy() const = 0;
/**
- * Gets this %Tokenizer's associated Numbers.
+ * Gets this %Tokenizer's associated State.
*
- * @return Returns said Numbers.
+ * @return Returns said State.
*/
- Numbers& numbers();
+ State& state();
/**
- * Gets this %Tokenizer's associated Numbers.
+ * Gets this %Tokenizer's associated State.
*
- * @return Returns said Numbers.
+ * @return Returns said State.
*/
- Numbers const& numbers() const;
+ State const& state() const;
/**
* Tokenizes the given node.
@@ -207,9 +208,9 @@
/**
* Constructs a %Tokenizer.
*
- * @param numbers the Numbers to use.
+ * @param state the State to use.
*/
- Tokenizer( Numbers &numbers );
+ Tokenizer( State &state );
/**
* Destroys a %Tokenizer.
@@ -255,18 +256,18 @@
Callback &callback, bool tokenize_acp );
private:
- Numbers *numbers_;
+ State *state_;
};
-inline Tokenizer::Tokenizer( Numbers &numbers ) : numbers_( &numbers ) {
-}
-
-inline Tokenizer::Numbers& Tokenizer::numbers() {
- return *numbers_;
-}
-
-inline Tokenizer::Numbers const& Tokenizer::numbers() const {
- return *numbers_;
+inline Tokenizer::Tokenizer( State &state ) : state_( &state ) {
+}
+
+inline Tokenizer::State& Tokenizer::state() {
+ return *state_;
+}
+
+inline Tokenizer::State const& Tokenizer::state() const {
+ return *state_;
}
inline void Tokenizer::tokenize_node( Item const &item,
@@ -288,13 +289,13 @@
* Creates a new %Tokenizer.
*
* @param lang The language of the text that the tokenizer will tokenize.
- * @param numbers The Numbers to use. If \c null, \a t is not set.
+ * @param state The State to use. If \c null, \a t is not set.
* @param t If not \c null, set to point to a Tokenizer for \a lang.
* @return Returns \c true only if this provider can provide a tokenizer for
* \a lang.
*/
virtual bool getTokenizer( locale::iso639_1::type lang,
- Tokenizer::Numbers *numbers = 0,
+ Tokenizer::State *state = 0,
Tokenizer::ptr *t = 0 ) const = 0;
};
=== modified file 'src/runtime/full_text/apply.cpp'
--- src/runtime/full_text/apply.cpp 2012-05-03 12:31:51 +
+++ src/runtime/full_text/apply.cpp 2012-05-16 00:57:21 +
@@ -1251,11 +1251,11 @@
FTTokenSeqIterator::FTTokens synonyms;
thesaurus_callback cb( qt0.pos(), qt0.lang(), synonyms );
- Tokenizer::Numbers t_num;
+ Tokenizer::State t_state;
TokenizerProvider const *const provider = GENV_STORE.getTokenizerProvider();
ZORBA_ASSERT( provider );
Tokenizer::ptr tokenizer;
- if ( !provider->getTokenizer( qt0.lang(), &t_num, &tokenizer ) )
+ if ( !provider->getTokenizer( qt0.lang(), &t_state, &tokenizer ) )
throw XQUERY_EXCEPTION(
err::FTST0009,
ERROR_PARAMS(
=== modified file 'src/runtime/full_text/ft_module_impl.cpp'
--- src/runtime/full_text/ft_module_impl.cpp 2012-05-15 21:13:21 +
+++
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue job feature-ft_module-2012-05-11T17-40-04.872Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-11T17-40-04.872Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
It doesn't. The '-' is part of the line, not the diff. -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Needs Information Why does the commit remove references to the stemmer, thesaurus, and tokenizer documentation? -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Paul J. Lucas has proposed merging lp:~zorba-coders/zorba/feature-ft_module into lp:zorba. Requested reviews: Matthias Brantner (matthias-brantner) Paul J. Lucas (paul-lucas) Related bugs: Bug #944795 in Zorba: "XQDoc doesn't handle & in URLs" https://bugs.launchpad.net/zorba/+bug/944795 For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 Added link to full-text module. Note that I have no way to test it. -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105506 Your team Zorba Coders is subscribed to branch lp:zorba. === modified file 'doc/zorba/ft_intro.dox' --- doc/zorba/ft_intro.dox 2012-05-03 12:31:51 + +++ doc/zorba/ft_intro.dox 2012-05-11 17:27:03 + @@ -8,6 +8,7 @@ - \ref ft_stemmer - \ref ft_thesaurus - \ref ft_tokenizer +- Full Text Module \section ft_unimplemented Unimplemented Features -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue job feature-ft_module-2012-05-11T02-50-07.325Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-11T02-50-07.325Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Needs review For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The attempt to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba failed. Below is the output from the failed tests. CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message): Validation queue job feature-ft_module-2012-05-11T02-08-17.708Z is finished. The final status was: 1 tests did not succeed - changes not commited. Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-11T02-08-17.708Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Paul J. Lucas has proposed merging lp:~zorba-coders/zorba/feature-ft_module
into lp:zorba.
Requested reviews:
Paul J. Lucas (paul-lucas)
Related bugs:
Bug #944795 in Zorba: "XQDoc doesn't handle & in URLs"
https://bugs.launchpad.net/zorba/+bug/944795
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421
Documentation tweaks.
--
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105421
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'doc/zorba/ft_tokenizer.dox'
--- doc/zorba/ft_tokenizer.dox 2012-05-03 12:31:51 +
+++ doc/zorba/ft_tokenizer.dox 2012-05-10 23:53:19 +
@@ -10,9 +10,12 @@
\section ft_tokenizer_tokization Tokenization
-Using the
-http://site.icu-project.org/";>ICU library,
-Zorba's implementation of tokenization
+By default,
+Zorba uses the
+http://site.icu-project.org/";>ICU library
+for tokenization.
+For Roman alphabets,
+Zorba (ICU)
considers only alpha-numeric sequences of characters to be part of a token;
whitespace and punctuation characters are not
and separate tokens.
@@ -117,7 +120,9 @@
\c lang
- The language of the string.
+ The
+ http://www.w3.org/TR/xmlschema-2/#language";>language
+ of the string.
@@ -132,7 +137,7 @@
\c callback
- The Callback to call once per token.
+ The \c Callback to call once per token.
@@ -232,7 +237,9 @@
\c languages
- The list of languages supported by the tokenizer.
+ The list of
+ http://www.w3.org/TR/xmlschema-2/#language";>languages
+ supported by the tokenizer.
@@ -247,7 +254,9 @@
In addition to a \c Tokenizer,
you must also implement a \c TokenizerProvider
-that, given a language, provides a \c Tokenizer for that language:
+that,
+given a http://www.w3.org/TR/xmlschema-2/#language";>language,
+provides a \c Tokenizer for that language:
\code
class TokenizerProvider {
@@ -262,7 +271,11 @@
\c lang
-The language to tokenize.
+
+ The
+ http://www.w3.org/TR/xmlschema-2/#language";>language
+ to tokenize.
+
\c num
--
Mailing list: https://launchpad.net/~zorba-coders
Post to : zorba-coders@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zorba-coders
More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105276 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105276 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue job feature-ft_module-2012-05-09T22-31-10.422Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105276 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-09T22-31-10.422Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105276 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105276 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105276 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105276 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105276 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105023 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105023 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue job feature-ft_module-2012-05-08T03-04-21.004Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105023 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-08T03-04-21.004Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105023 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105023 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105023 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105023 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105023 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Paul J. Lucas has proposed merging lp:~zorba-coders/zorba/feature-ft_module
into lp:zorba.
Requested reviews:
Paul J. Lucas (paul-lucas)
Matthias Brantner (matthias-brantner)
Related bugs:
Bug #944795 in Zorba: "XQDoc doesn't handle & in URLs"
https://bugs.launchpad.net/zorba/+bug/944795
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105023
Fixes the stop-words core dump on 64-bit Linux.
--
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/105023
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'src/runtime/full_text/ft_stop_words_set.cpp'
--- src/runtime/full_text/ft_stop_words_set.cpp 2012-05-03 12:31:51 +
+++ src/runtime/full_text/ft_stop_words_set.cpp 2012-05-08 02:54:58 +
@@ -17,14 +17,14 @@
#include
-#include
-#include
-#include
-#include
-#include
-#include
-#include
-#include
+#include "context/static_context.h"
+#include "context/uri_resolver.h"
+#include "util/ascii_util.h"
+#include "util/cxx_util.h"
+#include "util/mmap_file.h"
+#include "util/stl_util.h"
+#include "util/uri_util.h"
+#include "zorbautils/locale.h"
#include "ft_stop_words_set.h"
@@ -57,7 +57,7 @@
case LANG(pt);
case LANG(sv);
default:
- return 0;
+ return nullptr;
}
}
@@ -75,7 +75,6 @@
void ft_stop_words_set::apply_word( zstring const &word, word_set_t &word_set,
ft_stop_words_unex::type mode ) {
// TODO: should "word" be converted to lower-case?
- std::cout << "applying word " << word << std::endl;
switch ( mode ) {
case ft_stop_words_unex::union_:
word_set.insert( word );
@@ -106,12 +105,9 @@
must_delete = true;
break;
case ft_stop_words_mode::with_default:
- word_set = get_default_word_set_for( lang );
- if ( !word_set ) {
-// TODO: throw exception?
-return ptr();
- }
- break;
+ if ( (word_set = get_default_word_set_for( lang )) )
+break;
+ // no break;
case ft_stop_words_mode::without:
return ptr();
}
@@ -133,7 +129,7 @@
dynamic_cast( rsrc.get() );
if ( !stream_rsrc ) {
// Technically this should be thrown during static analysis.
-throw ZORBA_EXCEPTION( err::FTST0008, ERROR_PARAMS( uri ) );
+throw XQUERY_EXCEPTION( err::FTST0008, ERROR_PARAMS( uri ) );
}
std::istream *const stream = stream_rsrc->getStream();
=== modified file 'src/runtime/full_text/ft_token_matcher.cpp'
--- src/runtime/full_text/ft_token_matcher.cpp 2012-05-03 12:31:51 +
+++ src/runtime/full_text/ft_token_matcher.cpp 2012-05-08 02:54:58 +
@@ -47,12 +47,17 @@
return false;
}
-inline ft_stop_words_set::ptr
+//
+// This returns a raw pointer to work around a bug where ~ft_stop_words_set()
+// is being called when it shouldn't be due to ~unique_ptr() being called
+// without transferring ownership properly.
+//
+inline ft_stop_words_set const*
get_stop_words( ftmatch_options const &options, iso639_1::type lang,
static_context const& sctx ) {
if ( ftstop_word_option const *const sw = options.get_stop_word_option() )
-return ft_stop_words_set::construct( *sw, lang, sctx );
- return ft_stop_words_set::ptr();
+return ft_stop_words_set::construct( *sw, lang, sctx ).release();
+ return nullptr;
}
///
--
Mailing list: https://launchpad.net/~zorba-coders
Post to : zorba-coders@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zorba-coders
More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue job feature-ft_module-2012-05-07T15-29-00.759Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-07T15-29-00.759Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
> OK, so write the code that constructs the nodes with the right type > information. Not for 2.5. Doing this is a source of too many errors. -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
OK, so write the code that constructs the nodes with the right type information. -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
> I don't understand why the result doesn't have to be validated. Isn't that > what Dana requires? Simply because validation is too expensive. It takes about 45% of the time and that's not worth it. If we want it to be typed, we could already construct the nodes to have the right type information. -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
I don't understand why the result doesn't have to be validated. Isn't that what Dana requires? -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Needs review For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 2 Pending. -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue job feature-ft_module-2012-05-05T16-35-07.477Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-05T16-35-07.477Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Commit Message changed to: optimized ft:tokenize (no validation of tokens + factorized creation of qnames) For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Matthias Brantner has proposed merging lp:~zorba-coders/zorba/feature-ft_module
into lp:zorba.
Requested reviews:
Matthias Brantner (matthias-brantner)
Paul J. Lucas (paul-lucas)
Related bugs:
Bug #944795 in Zorba: "XQDoc doesn't handle & in URLs"
https://bugs.launchpad.net/zorba/+bug/944795
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841
optimized ft:tokenize (no validation of tokens + factorized creation of qnames)
--
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104841
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'src/runtime/full_text/ft_module_impl.cpp'
--- src/runtime/full_text/ft_module_impl.cpp 2012-04-28 00:48:03 +
+++ src/runtime/full_text/ft_module_impl.cpp 2012-05-05 16:30:27 +
@@ -453,9 +453,48 @@
///
+TokenizeIterator::TokenizeIterator(
+ static_context* sctx,
+ const QueryLoc& loc,
+ std::vector& children)
+ : NaryBaseIterator(sctx, loc, children)
+{
+ initMembers();
+}
+
+void TokenizeIterator::serialize( ::zorba::serialization::Archiver& ar)
+{
+ serialize_baseclass(ar,
+ (NaryBaseIterator*)this);
+ if (!ar.is_serializing_out())
+ {
+initMembers();
+ }
+}
+
+void TokenizeIterator::initMembers() {
+ GENV_ITEMFACTORY->createQName(
+token_qname_, static_context::ZORBA_FULL_TEXT_FN_NS, "", "token");
+
+ GENV_ITEMFACTORY->createQName(
+lang_qname_, "", "", "lang");
+
+ GENV_ITEMFACTORY->createQName(
+para_qname_, "", "", "paragraph");
+
+ GENV_ITEMFACTORY->createQName(
+sent_qname_, "", "", "sentence");
+
+ GENV_ITEMFACTORY->createQName(
+value_qname_, "", "", "value");
+
+ GENV_ITEMFACTORY->createQName(
+ref_qname_, "", "", "node-ref");
+}
+
bool TokenizeIterator::nextImpl( store::Item_t &result,
PlanState &plan_state ) const {
- store::Item_t attr_name, attr_node;
+ store::Item_t node_name, attr_node;
zstring base_uri;
store::Item_t item;
iso639_1::type lang;
@@ -488,69 +527,60 @@
token = state->doc_tokens_->next();
ZORBA_ASSERT( token );
- if ( state->token_qname_.isNull() )
-GENV_ITEMFACTORY->createQName(
- state->token_qname_, static_context::ZORBA_FULL_TEXT_FN_NS, "",
- "token"
-);
-
base_uri = static_context::ZORBA_FULL_TEXT_FN_NS;
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+ node_name = token_qname_;
GENV_ITEMFACTORY->createElementNode(
-result, nullptr, state->token_qname_, type_name, false, false,
+result, nullptr, node_name, type_name, false, false,
ns_bindings, base_uri
);
if ( token->lang() ) {
value_string = iso639_1::string_of[ token->lang() ];
-GENV_ITEMFACTORY->createQName( attr_name, "", "", "lang" );
GENV_ITEMFACTORY->createString( item, value_string );
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+node_name = lang_qname_;
GENV_ITEMFACTORY->createAttributeNode(
- attr_node, result, attr_name, type_name, item
+ attr_node, result, node_name, type_name, item
);
}
ztd::to_string( token->para(), &value_string );
- GENV_ITEMFACTORY->createQName( attr_name, "", "", "paragraph" );
GENV_ITEMFACTORY->createString( item, value_string );
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+ node_name = para_qname_;
GENV_ITEMFACTORY->createAttributeNode(
-attr_node, result, attr_name, type_name, item
+attr_node, result, node_name, type_name, item
);
ztd::to_string( token->sent(), &value_string );
- GENV_ITEMFACTORY->createQName( attr_name, "", "", "sentence" );
GENV_ITEMFACTORY->createString( item, value_string );
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+ node_name = sent_qname_;
GENV_ITEMFACTORY->createAttributeNode(
-attr_node, result, attr_name, type_name, item
+attr_node, result, node_name, type_name, item
);
value_string = token->value();
- GENV_ITEMFACTORY->createQName( attr_name, "", "", "value" );
GENV_ITEMFACTORY->createString( item, value_string );
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+ node_name = value_qname_;
GENV_ITEMFACTORY->createAttributeNode(
-attr_node, result, attr_name, type_name, item
+attr_node, result, node_name, type_name, item
);
if ( store::Item const *const token_item = token->item() ) {
if ( GENV_STORE.getNodeReference( item, token_item ) ) {
item->getStringValue2( value_string );
- GENV_ITEMFACTORY->createQName( attr_name, "", "", "node-ref" );
GENV_ITEMFACTORY->createString( item, value_string );
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+ node_name = ref_qname_;
GENV_ITEMFA
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Needs review For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104830 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104830 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The attempt to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba failed. Below is the output from the failed tests. CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message): Validation queue job feature-ft_module-2012-05-05T14-18-05.028Z is finished. The final status was: 3 tests did not succeed - changes not commited. Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104830 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-ft_module-2012-05-05T14-18-05.028Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104830 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104830 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104830 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104830 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
Matthias Brantner has proposed merging lp:~zorba-coders/zorba/feature-ft_module
into lp:zorba.
Requested reviews:
Paul J. Lucas (paul-lucas)
Related bugs:
Bug #944795 in Zorba: "XQDoc doesn't handle & in URLs"
https://bugs.launchpad.net/zorba/+bug/944795
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104830
optimized ft:tokenize (no validation of tokens + factorized creation of qnames)
--
https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/104830
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'src/runtime/full_text/ft_module_impl.cpp'
--- src/runtime/full_text/ft_module_impl.cpp 2012-04-28 00:48:03 +
+++ src/runtime/full_text/ft_module_impl.cpp 2012-05-05 11:42:19 +
@@ -453,9 +453,48 @@
///
+TokenizeIterator::TokenizeIterator(
+ static_context* sctx,
+ const QueryLoc& loc,
+ std::vector& children)
+ : NaryBaseIterator(sctx, loc, children)
+{
+ initMembers();
+}
+
+void TokenizeIterator::serialize( ::zorba::serialization::Archiver& ar)
+{
+ serialize_baseclass(ar,
+ (NaryBaseIterator*)this);
+ if (!ar.is_serializing_out())
+ {
+initMembers();
+ }
+}
+
+void TokenizeIterator::initMembers() {
+ GENV_ITEMFACTORY->createQName(
+token_qname_, static_context::ZORBA_FULL_TEXT_FN_NS, "", "token");
+
+ GENV_ITEMFACTORY->createQName(
+lang_qname_, "", "", "lang");
+
+ GENV_ITEMFACTORY->createQName(
+para_qname_, "", "", "paragraph");
+
+ GENV_ITEMFACTORY->createQName(
+sent_qname_, "", "", "sentence");
+
+ GENV_ITEMFACTORY->createQName(
+value_qname_, "", "", "value");
+
+ GENV_ITEMFACTORY->createQName(
+ref_qname_, "", "", "node-ref");
+}
+
bool TokenizeIterator::nextImpl( store::Item_t &result,
PlanState &plan_state ) const {
- store::Item_t attr_name, attr_node;
+ store::Item_t node_name, attr_node;
zstring base_uri;
store::Item_t item;
iso639_1::type lang;
@@ -488,69 +527,60 @@
token = state->doc_tokens_->next();
ZORBA_ASSERT( token );
- if ( state->token_qname_.isNull() )
-GENV_ITEMFACTORY->createQName(
- state->token_qname_, static_context::ZORBA_FULL_TEXT_FN_NS, "",
- "token"
-);
-
base_uri = static_context::ZORBA_FULL_TEXT_FN_NS;
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+ node_name = token_qname_;
GENV_ITEMFACTORY->createElementNode(
-result, nullptr, state->token_qname_, type_name, false, false,
+result, nullptr, node_name, type_name, false, false,
ns_bindings, base_uri
);
if ( token->lang() ) {
value_string = iso639_1::string_of[ token->lang() ];
-GENV_ITEMFACTORY->createQName( attr_name, "", "", "lang" );
GENV_ITEMFACTORY->createString( item, value_string );
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+node_name = lang_qname_;
GENV_ITEMFACTORY->createAttributeNode(
- attr_node, result, attr_name, type_name, item
+ attr_node, result, node_name, type_name, item
);
}
ztd::to_string( token->para(), &value_string );
- GENV_ITEMFACTORY->createQName( attr_name, "", "", "paragraph" );
GENV_ITEMFACTORY->createString( item, value_string );
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+ node_name = para_qname_;
GENV_ITEMFACTORY->createAttributeNode(
-attr_node, result, attr_name, type_name, item
+attr_node, result, node_name, type_name, item
);
ztd::to_string( token->sent(), &value_string );
- GENV_ITEMFACTORY->createQName( attr_name, "", "", "sentence" );
GENV_ITEMFACTORY->createString( item, value_string );
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+ node_name = sent_qname_;
GENV_ITEMFACTORY->createAttributeNode(
-attr_node, result, attr_name, type_name, item
+attr_node, result, node_name, type_name, item
);
value_string = token->value();
- GENV_ITEMFACTORY->createQName( attr_name, "", "", "value" );
GENV_ITEMFACTORY->createString( item, value_string );
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+ node_name = value_qname_;
GENV_ITEMFACTORY->createAttributeNode(
-attr_node, result, attr_name, type_name, item
+attr_node, result, node_name, type_name, item
);
if ( store::Item const *const token_item = token->item() ) {
if ( GENV_STORE.getNodeReference( item, token_item ) ) {
item->getStringValue2( value_string );
- GENV_ITEMFACTORY->createQName( attr_name, "", "", "node-ref" );
GENV_ITEMFACTORY->createString( item, value_string );
type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+ node_name = ref_qname_;
GENV_ITEMFACTORY->createAttributeNode(
-
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/feature-ft_module into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/feature-ft_module into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/103953 -- https://code.launchpad.net/~zorba-coders/zorba/feature-ft_module/+merge/103953 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
