Re: fts_solr not working

Sun, 26 Jun 2016 11:57:27 -0700 

On 2016-06-23 11:24, aki.tu...@dovecot.fi wrote:
On June 23, 2016 at 9:06 PM Jeff Gamsby <jeffgam...@merlock.com> wrote: 


I am running ISPConfig 3 on Debian and have managed to install the
dovecot-solr and dovecot-fts plugins.

I have solr running undet tomcat at http://localhost:8880 but the
indexing is not working.

I am using Dovecot 2.17


I do not understand namespaces and why fts_solr needs them, I just 
want

to index the entire users Maildir.


I am trying to index a users mailbox but am getting the following 
error:


(changed user name)
doveadm fts rescan -u u...@user.com inbox
doveadm(u...@user.com): Error: Namespace prefix not found: inbox

running that in debig mode gives:

doveadm(root): Debug: Loading modules from directory:
/usr/lib/dovecot/modules
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_fts_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib21_fts_solr_plugin.so
doveadm(root): Debug: Loading modules from directory:
/usr/lib/dovecot/modules/doveadm
doveadm(root): Debug: Skipping module doveadm_acl_plugin, because
dlopen() failed:

/usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: 
undefined

symbol: acl_user_module (this is usually intentional, so just ignore
this message)
doveadm(root): Debug: Skipping module doveadm_expire_plugin, because
dlopen() failed:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so:
undefined symbol: expire_set_deinit (this is usually intentional, so
just ignore this message)
doveadm(root): Debug: Skipping module doveadm_quota_plugin, because
dlopen() failed:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so:
undefined symbol: quota_user_module (this is usually intentional, so
just ignore this message)
doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because
dlopen() failed:

/usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so: 
undefined

symbol: i_stream_create_deflate (this is usually intentional, so just
ignore this message)
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so
doveadm(u...@user.com): Debug: Added userdb setting:
mail=maildir:/var/vmail/user.com/user/Maildir
doveadm(u...@user.com): Debug: Added userdb setting:
plugin/quota_rule=*:storage=0B
doveadm(u...@user.com): Debug: Added userdb setting:
plugin/sieve=/var/vmail/user.com/user/.sieve
doveadm(u...@user.com): Debug: Effective uid=5000, gid=5000,
home=/var/vmail/user.com/user
doveadm(u...@user.com): Debug: Namespace inbox: type=private, prefix=,
sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
location=maildir:/var/vmail/user.com/user/Maildir
doveadm(u...@user.com): Debug: maildir++:
root=/var/vmail/user.com/user/Maildir, index=, control=,
inbox=/var/vmail/user.com/user/Maildir, alt=
doveadm(u...@user.com): Debug: fts: No fts setting - plugin disabled
doveadm(u...@user.com): Error: Namespace prefix not found: inbox

in conf.d/10-mail.conf I have this namespace defined:

namespace inbox {
   # Namespace type: private, shared or public
   #type = private

   # Hierarchy separator to use. You should use the same separator for
all

   # namespaces or some clients get confused. '/' is usually a good 
one.
   # The default however depends on the underlying mail storage 
format.

   #separator =


   # Prefix required to access this namespace. This needs to be 
different

for
   # all namespaces. For example "Public/".
   #prefix =

   # Physical location of the mailbox. This is in same format as
   # mail_location, which is also the default for it.
   #location =

   # There can be only one INBOX, and this setting defines which
namespace
   # has it.
   inbox = yes


   # If namespace is hidden, it's not advertised to clients via 
NAMESPACE

   # extension. You'll most likely also want to set list=no. This is
mostly

   # useful when converting from another server with different 
namespaces

which
   # you want to deprecate but still keep working. For example you can
create

   # hidden namespaces with prefixes "~/mail/", "~%u/mail/" and 
"mail/".

   #hidden = no
protocol imap {
   plugin {
     fts = solr
     fts_solr = break-imap-search url=http://localhost:8880/solr/
   }
}
protocol pop3 {
   plugin {
     fts = solr
     fts_solr = break-imap-search url=http://localhost:8880/solr/
   }

   # Show the mailboxes under this namespace with LIST command. This
makes the
   # namespace visible for clients that don't support NAMESPACE
extension.
   # "children" value lists child mailboxes, but hides the namespace
prefix.
   #list = yes


   # Namespace handles its own subscriptions. If set to "no", the 
parent

   # namespace handles them (empty prefix should always have this as
"yes")
   #subscriptions = yes
}



I had to put this in dovecot.conf in order for the plugin to be 
enabled:


mail_plugins = fts fts_solr

solr is reachable at localhost:8880/solr and appears to be working.

Please help, any suggestions are welcome

Thanks


Can you please send doveconf -n?
---
Aki Tuomi




I managed to get fts_slor working and now I can index mailboxes, but I 
am getting a solr error and cannot use the indexes that were created


I get:

Error: fts_solr: Lookup failed: Internal Server Error


dovecot -n

# 2.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.1
auth_mechanisms = plain login
default_vsz_limit = 2 G
disable_plaintext_auth = no
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
listen = *,[::]
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_plugins = fts fts_solr
mail_privileged_group = vmail
managesieve_notify_capability = mailto

managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date ihave

namespace inbox {
  inbox = yes
  location =
  prefix =
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
plugin {
  fts = solr
  fts_autoindex = yes
  fts_solr = break-imap-search url=http://localhost:8880/solr/
  quota = dict:user::file:/var/vmail/%d/%n/.quotausage
  sieve = ~/.dovecot.sieve
  sieve_after = /etc/sieve/after
  sieve_before = /etc/sieve/before
  sieve_default = /var/vmail/sieve/default.sieve
  sieve_dir = ~/sieve
}
protocols = imap pop3
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
  unix_listener auth-master {
    group = vmail
    mode = 0660
    user = vmail
  }
  unix_listener auth-userdb {
    group = vmail
    mode = 0600
    user = vmail
  }
  user = root
}
service imap-login {
  client_limit = 1000
  process_limit = 500
}
service imap {
  vsz_limit = 2 G
}
ssl_cert = </etc/postfix/smtpd.cert
ssl_key = </etc/postfix/smtpd.key
userdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
protocol pop3 {
  pop3_uidl_format = %08Xu%08Xv
}
protocol lda {
  auth_socket_path = /var/run/dovecot/auth-master
  info_log_path = /var/log/dovecot-lda.log
  log_path = /var/log/dovecot-lda-errors.log
  mail_plugins = fts fts_solr autocreate sieve quota
  postmaster_address = postmas...@user.com
}



And this shows that indexes are correct:

doveadm -Dv index -u j...@user.com testing

doveadm(j...@user.com): Info: testing: Cache is already up to date



using solr-schema.xml from dovecot-2.2.13 sources in 
/etc/solr/conf/schema.xml


dovecot-sql.conf

# http://wiki.dovecot.org/AuthDatabase/SQL
#
# CREATE TABLE users (
#     userid VARCHAR(128) NOT NULL,
#     password VARCHAR(64) NOT NULL,
#     home VARCHAR(255) NOT NULL,
#     uid INTEGER NOT NULL,
#     gid INTEGER NOT NULL,
#     active CHAR(1) DEFAULT 'Y' NOT NULL
# );

driver = mysql

connect = host=localhost dbname=dbispconfig user=ispconfig 
password=xxxxxxxxxxxxxxxxxxxxx

default_pass_scheme = CRYPT


password_query = SELECT password FROM mail_user WHERE (login = '%u' OR 
email = '%u') AND `disable%Ls` = 'n' AND server_id = '1'
user_query = SELECT email as user, maildir as home, CONCAT('maildir:', 
maildir, '/Maildir') as mail, uid, gid, CONCAT('*:storage=', quota, 'B') 
AS quota_rule, CONCAT(maildir, '/.sieve') as sieve FROM mail_user WHERE 
(login = '%u' OR email = '%u') AND `disable%Ls` = 'n' AND server_id = 
'1'



# The iterate_query is required for the doveadm command only and works 
only on dovecot 2 servers.

# Do not enable it on Dovecot 1.x servers
iterate_query = SELECT email as user FROM mail_user






















					Reply via email to