[rt-users] docs improvement suggestion for full-text searching
So the documentation at https://www.bestpractical.com/docs/rt/4.2/UPGRADING-4.0.html says: > Since 4.0.0, RT's ticket content search is disabled by default because of > performance issues when used without full text indexing. For details on how > to re-enable it with (or without) full text indexing, see > docs/full_text_indexing.pod. However this URL https://www.bestpractical.com/docs/rt/4.2/full_text_indexing.html contains only information on how to enable full text indexing, and no information on how to enable it without the indexing. Seems like you need to add the information from http://blog.bestpractical.com/2011/06/full-text-searching.html to the top of this page. -- Jo Rhett +1 (415) 999-1798 Skype: jorhett Net Consonance : net philanthropy to improve open source and internet projects.
Re: [rt-users] docs improvement suggestion for full-text searching
On 11/26/2014 06:35 PM, Jo Rhett wrote: > However [doc/full_text_indexing] contains only information on how to > enable full text indexing, and no information on how to enable it > without the indexing. Mostly because we're afraid of users shooting themselves in the foot. Regardless, I've committed 2551dca5 which details how to do so. - Alex
Re: [rt-users] docs improvement suggestion for full-text searching
On Nov 27, 2014, at 6:55 AM, Alex Vandiver wrote: > On 11/26/2014 06:35 PM, Jo Rhett wrote: >> However [doc/full_text_indexing] contains only information on how to >> enable full text indexing, and no information on how to enable it >> without the indexing. > > Mostly because we're afraid of users shooting themselves in the foot. > Regardless, I've committed 2551dca5 which details how to do so. > - Alex Understood. Just a clarity nitpick :) Although I am confused by your statements that full text indexing isn’t available in MySQL. I’ve used this quite successfully in the past http://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html Likewise, MariaDB has Sphinx support compiled in and available in their packages, which made this process almost trivial to enable. https://mariadb.com/kb/en/mariadb/documentation/storage-engines/sphinx-storage-engine/about-sphinxse/ I found three problems in the config pushed out by the sbin/rt-setup-fulltext-index command: 1. You need to create and chown the var/sphinx directory it references mkdir /opt/rt4/var/sphinx chown -R sphinx:sphinx /opt/rt4/var/sphinx 2. You need to add this binlog path line binlog_path = /opt/rt4/var/sphinx 3. You need to add this compat line or searchd won’t start compat_sphinxql_magics = 0 Easy fixes… -- Jo Rhett +1 (415) 999-1798 Skype: jorhett Net Consonance : net philanthropy to improve open source and internet projects.
Re: [rt-users] docs improvement suggestion for full-text searching
On 11/28/2014 03:12 AM, Jo Rhett wrote: > Understood. Just a clarity nitpick :) Although I am confused by your > statements that full text indexing isn’t available in MySQL. I’ve used > this quite successfully in the past > http://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html At the time the documentation was written, MySQL only had FTS support using MyISAM, not InnoDB. Commits 84066c4 and 77641fc on the unmerged 4.2/mysql-native-fts branch clarify the topic, as well as implement native FTS support. > Likewise, MariaDB has Sphinx support compiled in and available in > their packages, which made this process almost trivial to enable. > https://mariadb.com/kb/en/mariadb/documentation/storage-engines/sphinx-storage-engine/about-sphinxse/ RT doesn't officially support MariaDB -- though this is almost purely because of lack of documentation and testing infrastucture, not because it is known to have failure modes. When MariaDB support is added, the Sphinx integration will be noted. > I found three problems in the config pushed out by the > sbin/rt-setup-fulltext-index command: > > 1. You need to create and chown the var/sphinx directory it references > mkdir /opt/rt4/var/sphinx > chown -R sphinx:sphinx /opt/rt4/var/sphinx Noted in > 2. You need to add this binlog path line >binlog_path = /opt/rt4/var/sphinx > > 3. You need to add this compat line or searchd won’t start > compat_sphinxql_magics = 0 What version of sphinx? Neither of those are necessary with 2.0. - Alex
Re: [rt-users] docs improvement suggestion for full-text searching
version 2.0.8-1 rpm package for EL6 On Dec 1, 2014, at 9:21 AM, Alex Vandiver wrote: > On 11/28/2014 03:12 AM, Jo Rhett wrote: >> Understood. Just a clarity nitpick :) Although I am confused by your >> statements that full text indexing isn’t available in MySQL. I’ve used >> this quite successfully in the past >> http://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html > > At the time the documentation was written, MySQL only had FTS support > using MyISAM, not InnoDB. Commits 84066c4 and 77641fc on the unmerged > 4.2/mysql-native-fts branch clarify the topic, as well as implement > native FTS support. > >> Likewise, MariaDB has Sphinx support compiled in and available in >> their packages, which made this process almost trivial to enable. >> https://mariadb.com/kb/en/mariadb/documentation/storage-engines/sphinx-storage-engine/about-sphinxse/ > > RT doesn't officially support MariaDB -- though this is almost purely > because of lack of documentation and testing infrastucture, not because > it is known to have failure modes. When MariaDB support is added, the > Sphinx integration will be noted. > >> I found three problems in the config pushed out by the >> sbin/rt-setup-fulltext-index command: >> >> 1. You need to create and chown the var/sphinx directory it references >> mkdir /opt/rt4/var/sphinx >> chown -R sphinx:sphinx /opt/rt4/var/sphinx > > Noted in > >> 2. You need to add this binlog path line >> binlog_path = /opt/rt4/var/sphinx >> >> 3. You need to add this compat line or searchd won’t start >> compat_sphinxql_magics = 0 > > What version of sphinx? Neither of those are necessary with 2.0. > - Alex > -- Jo Rhett +1 (415) 999-1798 Skype: jorhett Net Consonance : net philanthropy to improve open source and internet projects.
Re: [rt-users] docs improvement suggestion for full-text searching
On 12/01/2014 04:11 PM, Jo Rhett wrote: > version 2.0.8-1 rpm package for EL6 compat_sphinxql_magics was added in 2.0.1-beta, defaults to 0 in 2.1.1-beta, and was removed in 2.2.1-beta. I'm hesitant to add something to the documented configuration which will cause sphinx to fail on all other versions. Can you explain the failure mode of not having it more clearly? binlog_path was added in 1.10 -- I've added it, with a comment to the versions it's pertinent to. - Alex
Re: [rt-users] docs improvement suggestion for full-text searching
Sphinx refuses to run without that parameter. Which given that it wasn’t defined in the file they really should have set the default appropriately and not whined at the user, but this is the version of Sphinx currently in RHEL EPEL so there’s going to be a lot of RHEL/CentOS users running into this problem. On Dec 5, 2014, at 8:07 AM, Alex Vandiver wrote: > On 12/01/2014 04:11 PM, Jo Rhett wrote: >> version 2.0.8-1 rpm package for EL6 > > compat_sphinxql_magics was added in 2.0.1-beta, defaults to 0 in > 2.1.1-beta, and was removed in 2.2.1-beta. I'm hesitant to add > something to the documented configuration which will cause sphinx to > fail on all other versions. Can you explain the failure mode of not > having it more clearly? > > binlog_path was added in 1.10 -- I've added it, with a comment to the > versions it's pertinent to. > - Alex > -- Jo Rhett +1 (415) 999-1798 Skype: jorhett Net Consonance : net philanthropy to improve open source and internet projects.
Re: [rt-users] docs improvement suggestion for full-text searching
On 12/08/2014 01:12 PM, Jo Rhett wrote: > Sphinx refuses to run without that parameter. Which given that it > wasn’t defined in the file they really should have set the default > appropriately and not whined at the user, but this is the version of > Sphinx currently in RHEL EPEL so there’s going to be a lot of > RHEL/CentOS users running into this problem. I can't replicate the compat_sphinxql_magics problems you report with a stock Sphinx 2.0.8 from EPEL on CentOS 6. With a stock configuration as provided by 4.2-trunk, indexer runs with no errors (see below). If it refuses to run, please show your configuration file, the sphinx version, and the actual error when running without compat_sphinxql_magics. - Alex -bash-4.1# rm /opt/rt4/var/sphinx/* -bash-4.1# indexer --config tmp.conf rt Sphinx 2.0.8-id64-release (r3831) Copyright (c) 2001-2012, Andrew Aksyonoff Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com) using config file 'tmp.conf'... indexing index 'rt'... WARNING: Attribute count is 0: switching to none docinfo collected 1 docs, 0.0 MB sorted 0.0 Mhits, 100.0% done total 1 docs, 626 bytes total 0.009 sec, 64073 bytes/sec, 102.35 docs/sec total 2 reads, 0.000 sec, 0.3 kb/call avg, 0.0 msec/call avg total 6 writes, 0.000 sec, 0.3 kb/call avg, 0.0 msec/call avg -bash-4.1# cat tmp.conf source rt { type= mysql sql_host= 127.0.0.1 sql_db = rt4 sql_user= root sql_pass= sql_query_pre = SET NAMES utf8 sql_query = \ SELECT a.id, a.content FROM Attachments a \ JOIN Transactions txn ON a.TransactionId = txn.id AND txn.ObjectType = 'RT::Ticket' \ JOIN Tickets t ON txn.ObjectId = t.id \ WHERE a.ContentType = 'text/plain' AND t.Status != 'deleted' sql_query_info = SELECT * FROM Attachments WHERE id=$id } index rt { source = rt path= /opt/rt4/var/sphinx/index docinfo = extern charset_type= utf-8 } indexer { mem_limit = 32M } searchd { port= 3312 log = /opt/rt4/var/sphinx/searchd.log query_log = /opt/rt4/var/sphinx/query.log read_timeout= 5 max_children= 30 pid_file= /opt/rt4/var/sphinx/searchd.pid max_matches = 1000 seamless_rotate = 1 preopen_indexes = 0 unlink_old = 1 }
Re: [rt-users] docs improvement suggestion for full-text searching
On 12/08/2014 01:12 PM, Jo Rhett wrote: >> Sphinx refuses to run without that parameter. Which given that it >> wasn’t defined in the file they really should have set the default >> appropriately and not whined at the user, but this is the version of >> Sphinx currently in RHEL EPEL so there’s going to be a lot of >> RHEL/CentOS users running into this problem. On Dec 8, 2014, at 11:24 AM, Alex Vandiver wrote: > I can't replicate the compat_sphinxql_magics problems you report with a > stock Sphinx 2.0.8 from EPEL on CentOS 6. With a stock configuration as > provided by 4.2-trunk, indexer runs with no errors (see below). If it RT's use of sphinx requires the daemon to answer, which you didn’t start. With your example file when I start the daemon I get this error: # service searchd start Starting searchd: Sphinx 2.0.8-id64-release (r3831) Copyright (c) 2001-2012, Andrew Aksyonoff Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/sphinx/sphinx.conf'... WARNING: compat_sphinxql_magics=1 is deprecated; please update your application and config Note that your file doesn’t contain magics=1. So I totally agree that it’s an annoying bug that the developers should fix, but it’s also something you can avoid by putting it in the file you output. -- Jo Rhett +1 (415) 999-1798 Skype: jorhett Net Consonance : net philanthropy to improve open source and internet projects.
Re: [rt-users] docs improvement suggestion for full-text searching
On 12/14/2014 08:25 PM, Jo Rhett wrote: > RT's use of sphinx requires the daemon to answer, which you didn’t > start. With your example file when I start the daemon I get this > error: > > # service searchd start Starting searchd: Sphinx 2.0.8-id64-release > (r3831) Copyright (c) 2001-2012, Andrew Aksyonoff Copyright (c) > 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com) > > using config file '/etc/sphinx/sphinx.conf'... WARNING: > compat_sphinxql_magics=1 is deprecated; please update your > application and config As the output notes, that is merely a warning. It is not an error. searchd starts up just fine for me: # service searchd start Starting searchd: Sphinx 2.0.8-id64-release (r3831) Copyright (c) 2001-2012, Andrew Aksyonoff Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/sphinx/sphinx.conf'... WARNING: compat_sphinxql_magics=1 is deprecated; please update your application and config listening on all interfaces, port=3312 precaching index 'rt' precached 1 indexes in 0.001 sec [ OK ] # service searchd status searchd (pid 15041) is running... > Note that your file doesn’t contain magics=1. So I totally agree that > it’s an annoying bug that the developers should fix, but it’s also > something you can avoid by putting it in the file you output. Since it starts up fine without the line, and would fail to start on Sphinx < 2 or > 2.2 with the line, I'm unconvinced on adding it to the default configuration. Administrators who are running Sphinx 2.0.x can simply heed the warning and add compat_sphinxql_magics=0 to their configuration. - Alex