Re: problem in setting up proxy

[Aki Tuomi via dovecot]

Hi!

First of all, can you provide output of 'doveconf -n'? It's much cleaner
to read and shows what's really there?

Aki

On 22.1.2019 17.57, Ted wrote:
> Hello,
>
> We're having difficulty with our updated cluster of dovecot servers
> accessing the email storage on the NFS mounts.  It seems index files get
> corrupted when 2 backend mailservers access the same account, and from
> documentation setting up a director proxy in front of the backup
> servers.  I'm trying to just set up a straight proxy first, which the
> documents say is the first step, and although I can see the connections
> coming into the server when I try to login via the proxy, the connection
> times out and there are no logs from dovecot anywhere saying what
> happened to the connection.
>
> The configs I have set up for this in dovecot are:
>
> dovecot.conf
>
> # Protocols we want to be serving.
> protocols = imap pop3
>
> #when re-enabling quota enforcement add quota in below:
> mail_plugins = $mail_plugins mail_log notify
>
> protocol imap {
>   # Space separated list of plugins to load (default is global
> mail_plugins).
> #when re-enabling quota enforcement add imap_quota in below:
>   mail_plugins = $mail_plugins
> }
>
>
> # A comma separated list of IPs or hosts where to listen in for
> connections.
> # "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces.
> # If you want to specify non-default ports or anything more complex,
> # edit conf.d/master.conf.
> #listen = *, ::
>
> # Base directory where to store runtime data.
> #base_dir = /var/run/dovecot/
>
> # Name of this instance. In multi-instance setup doveadm and other commands
> # can use -i  to select which instance is used (an
> alternative
> # to -c ). The instance name is also added to Dovecot processes
> # in ps output.
> #instance_name = dovecot
>
> # Greeting message for clients.
> login_greeting = Welcome to easyMail.
>
> shutdown_clients = yes
>
> # Most of the actual configuration gets included below. The filenames are
> # first sorted by their ASCII value and parsed in that order. The
> 00-prefixes
> # in filenames are intended to make it easier to understand the ordering.
> !include conf.d/*.conf
>
> # A config file can also tried to be included without giving an error if
> # it's not found:
> !include_try local.conf
>
> service auth {
>   unix_listener auth-master {
>     mode = 0600
>     user = vmail
>   }
> }
>
> conf.d/10-auth.conf
>
> ##
> ## Authentication processes
> ##
> # Username formatting before it's looked up from databases. You can use
> # the standard variables here, eg. %Lu would lowercase the username, %n
> would
> # drop away the domain if it was given, or "%n-AT-%d" would change the
> '@' into
> # "-AT-". This translation is done after auth_username_translation changes.
> auth_username_format = %Lu
>
> # Space separated list of wanted authentication mechanisms:
> #   plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp skey
> #   gss-spnego
> # NOTE: See also disable_plaintext_auth setting.
> auth_mechanisms = plain login
> #
> # Password database is used to verify user's password (and nothing more).
> # You can have multiple passdbs and userdbs. This is useful if you want to
> # allow both system users (/etc/passwd) and virtual users to login without
> # duplicating the system users into virtual database.
> #
> # 
> #
> # User database specifies where mails are located and what user/group IDs
> # own them. For single-UID configuration use "static" userdb.
> #
> # 
>
> #!include auth-deny.conf.ext
> #!include auth-master.conf.ext
>
> #!include auth-system.conf.ext
> #!include auth-sql.conf.ext
> #!include auth-ldap.conf.ext
> #!include auth-passwdfile.conf.ext
> #!include auth-checkpassword.conf.ext
> #!include auth-vpopmail.conf.ext
> !include auth-static.conf.ext
>
> conf.d/auth-static.conf.ext
>
> # Static passdb. Included from auth.conf.
>
> # This can be used for situations where Dovecot doesn't need to verify the
> # username or the password, or if there is a single password for all users:
> #
> #  - proxy frontend, where the backend verifies the password
> #  - proxy backend, where the frontend already verified the password
> #  - authentication with SSL certificates
> #  - simple testing
>
>   passdb static {
>    driver = static
>    args = nopassword=y
>    default_fields = proxy=y host=10.5.10.121
>   }
>
>
> #passdb {
> #  driver = static
> #  args = password=test
> #}
>
> #userdb {
> #  driver = static
> #  args = uid=vmail gid=vmail home=/home/%u
> #}
>
> conf.d/10-logging.conf
>
> ##
> ## Log destination.
> ##
>
> # Log file to use for error messages. "syslog" logs to syslog,
> # /dev/stderr logs to stderr.
> #log_path = syslog
>
> # Log file to use for informational messages. Defaults to log_path.
> #info_log_path =
> # Log file to use for debug messages. Defaults to info_log_path.
> #debug_log_path =
>
> # Syslog facility to use if you're logging to syslog. Usually if you don't
> # want to use 

Re: debian10+dovecot-2.2.33.2

[Aki Tuomi via dovecot]

Maybe you should try 2.2.36?

Aki

On 22.1.2019 16.47, Maciej Milaszewski IQ PL wrote:
> Hi
> I have little problem with debian10 and dovecot 2.2.33.2
>
> ps -ax
> 21815 ?    S  0:00 dovecot/pop3-login director
> 21816 ?    S  0:00 dovecot/pop3-login director
> 21817 ?    S  0:00 dovecot/pop3-login director
> 21818 ?    S  0:00 dovecot/pop3-login director
> 21819 ?    S  0:00 dovecot/pop3-login director
> 21821 ?    S  0:00 dovecot/pop3-login director
> 21822 ?    S  0:00 dovecot/pop3-login director
>
> But if I testes via telnet like this:
>
> telnet 46.xxx.xxx.113 143
> Trying 46.xxx.xxx.113...
> Connected to 46.xxx.xxx.113.
> Escape character is '^]'.
> * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE
> IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Imap ready.
> ^]
> telnet> q
> Connection closed.
>
> and ps:
> 21808 ?    S  0:00 dovecot/imap-login  
> ???
> 21826 ?    S  0:00 dovecot/imap-login  
> ?
>
> strace:
> strace -p 21808
> strace: Process 21808 attached
> gettimeofday({tv_sec=1548168193, tv_usec=419420}, NULL) = 0
> epoll_wait(14,
>
> doveconf:
> doveconf -n |head -n 4
> # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.4.8 (0c4ae064f307+)
> # OS: Linux 4.19.0-1-amd64 x86_64 Debian buster/sid
> auth_cache_negative_ttl = 5 mins
>
>
>
> Probably dovecot not closed correctly imap-login and this same for
> imap-pop3. Any idea ?
>
>

problem in setting up proxy

[Ted]

Hello,

We're having difficulty with our updated cluster of dovecot servers
accessing the email storage on the NFS mounts.  It seems index files get
corrupted when 2 backend mailservers access the same account, and from
documentation setting up a director proxy in front of the backup
servers.  I'm trying to just set up a straight proxy first, which the
documents say is the first step, and although I can see the connections
coming into the server when I try to login via the proxy, the connection
times out and there are no logs from dovecot anywhere saying what
happened to the connection.

The configs I have set up for this in dovecot are:

dovecot.conf

# Protocols we want to be serving.
protocols = imap pop3

#when re-enabling quota enforcement add quota in below:
mail_plugins = $mail_plugins mail_log notify

protocol imap {
  # Space separated list of plugins to load (default is global
mail_plugins).
#when re-enabling quota enforcement add imap_quota in below:
  mail_plugins = $mail_plugins
}


# A comma separated list of IPs or hosts where to listen in for
connections.
# "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces.
# If you want to specify non-default ports or anything more complex,
# edit conf.d/master.conf.
#listen = *, ::

# Base directory where to store runtime data.
#base_dir = /var/run/dovecot/

# Name of this instance. In multi-instance setup doveadm and other commands
# can use -i  to select which instance is used (an
alternative
# to -c ). The instance name is also added to Dovecot processes
# in ps output.
#instance_name = dovecot

# Greeting message for clients.
login_greeting = Welcome to easyMail.

shutdown_clients = yes

# Most of the actual configuration gets included below. The filenames are
# first sorted by their ASCII value and parsed in that order. The
00-prefixes
# in filenames are intended to make it easier to understand the ordering.
!include conf.d/*.conf

# A config file can also tried to be included without giving an error if
# it's not found:
!include_try local.conf

service auth {
  unix_listener auth-master {
    mode = 0600
    user = vmail
  }
}

conf.d/10-auth.conf

##
## Authentication processes
##
# Username formatting before it's looked up from databases. You can use
# the standard variables here, eg. %Lu would lowercase the username, %n
would
# drop away the domain if it was given, or "%n-AT-%d" would change the
'@' into
# "-AT-". This translation is done after auth_username_translation changes.
auth_username_format = %Lu

# Space separated list of wanted authentication mechanisms:
#   plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp skey
#   gss-spnego
# NOTE: See also disable_plaintext_auth setting.
auth_mechanisms = plain login
#
# Password database is used to verify user's password (and nothing more).
# You can have multiple passdbs and userdbs. This is useful if you want to
# allow both system users (/etc/passwd) and virtual users to login without
# duplicating the system users into virtual database.
#
# 
#
# User database specifies where mails are located and what user/group IDs
# own them. For single-UID configuration use "static" userdb.
#
# 

#!include auth-deny.conf.ext
#!include auth-master.conf.ext

#!include auth-system.conf.ext
#!include auth-sql.conf.ext
#!include auth-ldap.conf.ext
#!include auth-passwdfile.conf.ext
#!include auth-checkpassword.conf.ext
#!include auth-vpopmail.conf.ext
!include auth-static.conf.ext

conf.d/auth-static.conf.ext

# Static passdb. Included from auth.conf.

# This can be used for situations where Dovecot doesn't need to verify the
# username or the password, or if there is a single password for all users:
#
#  - proxy frontend, where the backend verifies the password
#  - proxy backend, where the frontend already verified the password
#  - authentication with SSL certificates
#  - simple testing

  passdb static {
   driver = static
   args = nopassword=y
   default_fields = proxy=y host=10.5.10.121
  }


#passdb {
#  driver = static
#  args = password=test
#}

#userdb {
#  driver = static
#  args = uid=vmail gid=vmail home=/home/%u
#}

conf.d/10-logging.conf

##
## Log destination.
##

# Log file to use for error messages. "syslog" logs to syslog,
# /dev/stderr logs to stderr.
#log_path = syslog

# Log file to use for informational messages. Defaults to log_path.
#info_log_path =
# Log file to use for debug messages. Defaults to info_log_path.
#debug_log_path =

# Syslog facility to use if you're logging to syslog. Usually if you don't
# want to use "mail", you'll use local0..local7. Also other standard
# facilities are supported.
#syslog_facility = mail

##
## Logging verbosity and debugging.
##

# Log unsuccessful authentication attempts and the reasons why they failed.
auth_verbose = yes

# In case of password mismatches, log the attempted password. Valid
values are
# no, plain and sha1. sha1 can be useful for detecting brute force password
# attempts vs. user simply trying 

debian10+dovecot-2.2.33.2

[Maciej Milaszewski IQ PL]

Hi
I have little problem with debian10 and dovecot 2.2.33.2

ps -ax
21815 ?    S  0:00 dovecot/pop3-login director
21816 ?    S  0:00 dovecot/pop3-login director
21817 ?    S  0:00 dovecot/pop3-login director
21818 ?    S  0:00 dovecot/pop3-login director
21819 ?    S  0:00 dovecot/pop3-login director
21821 ?    S  0:00 dovecot/pop3-login director
21822 ?    S  0:00 dovecot/pop3-login director

But if I testes via telnet like this:

telnet 46.xxx.xxx.113 143
Trying 46.xxx.xxx.113...
Connected to 46.xxx.xxx.113.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE
IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Imap ready.
^]
telnet> q
Connection closed.

and ps:
21808 ?    S  0:00 dovecot/imap-login  
???
21826 ?    S  0:00 dovecot/imap-login  
?

strace:
strace -p 21808
strace: Process 21808 attached
gettimeofday({tv_sec=1548168193, tv_usec=419420}, NULL) = 0
epoll_wait(14,

doveconf:
doveconf -n |head -n 4
# 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.8 (0c4ae064f307+)
# OS: Linux 4.19.0-1-amd64 x86_64 Debian buster/sid
auth_cache_negative_ttl = 5 mins



Probably dovecot not closed correctly imap-login and this same for
imap-pop3. Any idea ?


-- 
Maciej Miłaszewski
IQ PL Sp. z o.o.
Starszy Administrator Systemowy

Biuro Obsługi Klienta:
e-mail: b...@iq.pl
tel.: +48 58 326 09 90 - 94
fax: +48 58 326 09 99

Dział pomocy: https://www.iq.pl/pomoc
Informacja dotycząca przetwarzania danych osobowych: https://www.iq.pl/kontakt
Jakość gwarantuje: ISO 9001:2000

IQ PL Sp. z o.o. z siedzibą w Gdańsku (80-298), ul. Geodetów 16, KRS 
007725, Sąd rejestrowy: Sąd Rejonowy w Gdańsku VII Wydział KRS, 
kapitał zakładowy: 140.000 PLN, NIP 5832736211, REGON 192478853

Re: Moving messages between servers with different configurations

[Sami Ketola]

Hi,

not easily. You can always limit dsync to a single folder with -m folder, but 
then again it's designed to make 1:1 copy so you would need to rename the 
synced folder afterwards.

Sami

> On 22 Jan 2019, at 17.55, Marc Roos  wrote:
> 
> 
> This pulls over the whole test mailbox, but I need it to be stored in a 
> different mailbox, of course that is now possible to move from the local 
> test to the local INBOX/test2. But then I might aswell scp the mbox in 
> the first place. 
> 
> Should we conclude that it is not possible with a single doveadm command 
> to move messages from RemoteServerA/usertest/mailboxA/messagesof2017 to 
> LocalServerB/usertest/mailboxX/ ?
> 
> [@~]# doveadm mailbox list -u testuser
> Drafts
> Junk
> Trash
> INBOX/test2
> INBOX/test3
> Sent
> Archive
> Archive/Archive
> Archive/2019old
> INBOX
> [@~]# doveadm backup -u testuser -R -m test tcp:192.168.10.43:542
> [@~]# doveadm mailbox list -u testuser
> Drafts
> Junk
> Trash
> INBOX/test2
> INBOX/test3
> Sent
> test   <--- new!
> Archive
> Archive/Archive
> Archive/2019old
> INBOX
> 
>> try running this on the NEW dovecot server:
>> 
>> doveadm backup -u uid -R tcp:192.168.10.43:542
>> 
>> to pull the messages from old server.
>> 
>>> I feel a bit like an idiot, but I have been trying with copy, so I 
> do 
>>> not lose any message when testing. But I can't get them to copy. I 
> do 
>>> indeed have same uid and gid.
>>> 
>>> 
>>> [@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
>>> messages test
>>> messages=43
>>> [@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
>>> messages=16
>>> 
>>> [@~]# doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 
>>> mailbox test
>>> [@~]#
>>> 
>>> [@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
>>> messages test
>>> messages=43
>>> [@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
>>> messages=16 
>>> 
>>> Also tried with
>>> doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 user 
>>> testuser mailbox test
>>> 
>>> 
>>> 
 doveadm [-Dv] move [-S socket_path] -u user  destination [user 
>>> source_user] search_query
 
 Moving all mails from mailbox INBOX/test on serverA to mailbox
 Archive/2017 on local serverB.
 + destination Archive/2017 must exist
 + Limitation: source_user and testuser must share the same UID and 
>>> GID
 
 doveadm move -S x.x.x.x:x -u testuserAtServerB
 Archive/2017 user source_userAtServerA mailbox INBOX/test ALL
 
 
> I wanted to move messages from a mbox mailbox on server A to mdbox
> on 
> server B. I thought I could do this by connecting to the remote
> server 
> with "doveadm move -S x.x.x.x:x -u testuser Archive/2017 mailbox 
> INBOX/test" but I guess this will only allow and move messages 
> internally on server B?
> Should I use dsync, or is there another way to move the messages?
> 

RE: Having temp mdbox files, what to do with them?

[Marc Roos]

 

I assume yes because the X-Universally-Unique-Identifier, I can all find 
in the original mailbox.

 >Can I delete these because doveadm is only purging from source, when 
it 
 >is confirmed they are correctly at the destination?
 >
 >
 >[@ storage]# ls -ac1rt
 >m.554
 >m.569
 >m.564
 >m.559
 >m.561
 >.temp.1548002572.P1969Q0M313023.mail03
 >.temp.1548002572.P1969Q1M313023.mail03
 >.temp.1548002572.P1969Q2M313023.mail03
 >.
 >.temp.1548002572.P1969Q3M313023.mail03
 >
 >

Having temp mdbox files, what to do with them?

[Marc Roos]

Can I delete these because doveadm is only purging from source, when it 
is confirmed they are correctly at the destination?


[@ storage]# ls -ac1rt
m.554
m.569
m.564
m.559
m.561
.temp.1548002572.P1969Q0M313023.mail03
.temp.1548002572.P1969Q1M313023.mail03
.temp.1548002572.P1969Q2M313023.mail03
.
.temp.1548002572.P1969Q3M313023.mail03

RE: Moving messages between servers with different configurations

[Marc Roos]

 
This pulls over the whole test mailbox, but I need it to be stored in a 
different mailbox, of course that is now possible to move from the local 
test to the local INBOX/test2. But then I might aswell scp the mbox in 
the first place. 

Should we conclude that it is not possible with a single doveadm command 
to move messages from RemoteServerA/usertest/mailboxA/messagesof2017 to 
LocalServerB/usertest/mailboxX/ ?

[@~]# doveadm mailbox list -u testuser
Drafts
Junk
Trash
INBOX/test2
INBOX/test3
Sent
Archive
Archive/Archive
Archive/2019old
INBOX
[@~]# doveadm backup -u testuser -R -m test tcp:192.168.10.43:542
[@~]# doveadm mailbox list -u testuser
Drafts
Junk
Trash
INBOX/test2
INBOX/test3
Sent
test   <--- new!
Archive
Archive/Archive
Archive/2019old
INBOX

 >try running this on the NEW dovecot server:
 >
 >doveadm backup -u uid -R tcp:192.168.10.43:542
 >
 >to pull the messages from old server.
 >
 >> I feel a bit like an idiot, but I have been trying with copy, so I 
do 
 >> not lose any message when testing. But I can't get them to copy. I 
do 
 >> indeed have same uid and gid.
 >> 
 >> 
 >> [@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
 >> messages test
 >> messages=43
 >> [@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
 >> messages=16
 >> 
 >> [@~]# doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 
 >> mailbox test
 >> [@~]#
 >> 
 >> [@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
 >> messages test
 >> messages=43
 >> [@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
 >> messages=16 
 >> 
 >> Also tried with
 >> doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 user 
 >> testuser mailbox test
 >> 
 >> 
 >> 
 >>> doveadm [-Dv] move [-S socket_path] -u user  destination [user 
 >> source_user] search_query
 >>> 
 >>> Moving all mails from mailbox INBOX/test on serverA to mailbox
 >>> Archive/2017 on local serverB.
 >>> + destination Archive/2017 must exist
 >>> + Limitation: source_user and testuser must share the same UID and 
 >> GID
 >>> 
 >>> doveadm move -S x.x.x.x:x -u testuserAtServerB
 >>> Archive/2017 user source_userAtServerA mailbox INBOX/test ALL
 >>> 
 >>> 
  I wanted to move messages from a mbox mailbox on server A to mdbox
  on 
  server B. I thought I could do this by connecting to the remote
  server 
  with "doveadm move -S x.x.x.x:x -u testuser Archive/2017 mailbox 
  INBOX/test" but I guess this will only allow and move messages 
  internally on server B?
  Should I use dsync, or is there another way to move the messages?
  

Re: Moving messages between servers with different configurations

[Sami Ketola]

Hi,

try running this on the NEW dovecot server:

doveadm backup -u uid -R tcp:192.168.10.43:542

to pull the messages from old server.

Sami


> On 22 Jan 2019, at 16.52, Marc Roos  wrote:
> 
> 
> 
> Hi Martin,
> 
> I feel a bit like an idiot, but I have been trying with copy, so I do 
> not lose any message when testing. But I can't get them to copy. I do 
> indeed have same uid and gid.
> 
> 
> [@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
> messages test
> messages=43
> [@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
> messages=16
> 
> [@~]# doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 
> mailbox test
> [@~]#
> 
> [@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
> messages test
> messages=43
> [@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
> messages=16 
> 
> Also tried with
> doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 user 
> testuser mailbox test
> 
> 
> 
>> doveadm [-Dv] move [-S socket_path] -u user  destination [user 
> source_user] search_query
>> 
>> Moving all mails from mailbox INBOX/test on serverA to mailbox
>> Archive/2017 on local serverB.
>> + destination Archive/2017 must exist
>> + Limitation: source_user and testuser must share the same UID and 
> GID
>> 
>> doveadm move -S x.x.x.x:x -u testuserAtServerB
>> Archive/2017 user source_userAtServerA mailbox INBOX/test ALL
>> 
>> 
>>> I wanted to move messages from a mbox mailbox on server A to mdbox
>>> on 
>>> server B. I thought I could do this by connecting to the remote
>>> server 
>>> with "doveadm move -S x.x.x.x:x -u testuser Archive/2017 mailbox 
>>> INBOX/test" but I guess this will only allow and move messages 
>>> internally on server B?
>>> Should I use dsync, or is there another way to move the messages?
>>> 
>>> 
>>> 
> 

RE: Moving messages between servers with different configurations

[Marc Roos]

Hi Martin,

I feel a bit like an idiot, but I have been trying with copy, so I do 
not lose any message when testing. But I can't get them to copy. I do 
indeed have same uid and gid.


[@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
messages test
messages=43
[@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
messages=16

[@~]# doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 
mailbox test
[@~]#

[@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
messages test
messages=43
[@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
messages=16 

Also tried with
doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 user 
testuser mailbox test



 >doveadm [-Dv] move [-S socket_path] -u user  destination [user 
source_user] search_query
 >
 >Moving all mails from mailbox INBOX/test on serverA to mailbox
 >Archive/2017 on local serverB.
 > + destination Archive/2017 must exist
 > + Limitation: source_user and testuser must share the same UID and 
GID
 >
 >doveadm move -S x.x.x.x:x -u testuserAtServerB
 > Archive/2017 user source_userAtServerA mailbox INBOX/test ALL
 >
 >
 >> I wanted to move messages from a mbox mailbox on server A to mdbox
 >> on 
 >> server B. I thought I could do this by connecting to the remote
 >> server 
 >> with "doveadm move -S x.x.x.x:x -u testuser Archive/2017 mailbox 
 >> INBOX/test" but I guess this will only allow and move messages 
 >> internally on server B?
 >> Should I use dsync, or is there another way to move the messages?
 >> 
 >> 
 >> 

Re: Moving messages between servers with different configurations

[Martin Johannes Dauser]

Hi,

doveadm [-Dv] move [-S socket_path] -u user
 destination [user
source_user] search_query

Moving all mails from mailbox INBOX/test on serverA to mailbox
Archive/2017 on local serverB.
 + destination Archive/2017 must exist
 + Limitation: source_user and testuser must share the same UID and GID

doveadm move -S x.x.x.x:x -u testuserAtServerB 
 Archive/2017 user source_userAtServerA mailbox INBOX/test ALL


Greetings
Martin Johannes Dauser


On Mon, 2019-01-21 at 23:14 +0100, Marc Roos wrote:
> I wanted to move messages from a mbox mailbox on server A to mdbox
> on 
> server B. I thought I could do this by connecting to the remote
> server 
> with "doveadm move -S x.x.x.x:x -u testuser Archive/2017 mailbox 
> INBOX/test" but I guess this will only allow and move messages 
> internally on server B?
> Should I use dsync, or is there another way to move the messages?
> 
> 
> 

Encoding input

[Joan Moreau via dovecot]

Hi 


Are data inputs of fts_backend_update_build_more(struct
fts_backend_update_context *_ctx, const unsigned char *data, size_t
size) already converted to UTF8 ? 


Thanks

Re: Changing the imaps port #

[Michael A. Peters]

Another possible thing, I don't know what the bug is or if it is fixed, 
but few years ago Thunderbird (on CentOS 7) for me refused to connect to 
Port 993 or Port 465 if I used a self-signed certificate even though the 
same certificate worked when using STARTTLS and port 143 and 587. The 
error wasn't an SSL error, it just would act like it was not connecting.


With CA signed certificates it did work on 993 and 465.

On 1/22/19 1:25 AM, Michael A. Peters wrote:
Comcast DNS servers enforce dnssec, AT does not (last I checked). If 
by chance your zone has DNSSEC enabled but mis-configured then it is 
possible the domain name you use for the dovecot server is not resolving 
because of a dnssec validation failure.


I have never heard of comcast or any ISP blocking port 993. That would 
seem to be a violation of net neutrality rules. I use comcast (consumer, 
not business) and it does not block 993 (does block 25 but that it 
should block for dynamic issued addresses)


Look at the domain name used in your e-mail client and make sure it 
actually resolves. If it does not, check to see if DNSSEC validation is 
the issue.


On 1/21/19 8:58 PM, Patrick Mahan wrote:
Yes, I am pretty sure about that.  I originally was connected via AT 
DSL but wanted the fast access of cable modem.  I need permanent IPs 
which required me to contract with Comcast buisness.  Once I switched 
over, I was no longer able to access my imap server, which was as I 
mentioned, stunnel listening on the imaps port and forwarding to 
dovecot listening on the imap port.


I was getting connection refused on my laptop (thunderbird) email 
client when I was not at home.  I validated that it was not because it 
was reaching my email server.  So who ever was rejecting it, I assumed 
it was somewhere inside the comcast network.  Once I switch to a 
non-standard port, I was able to connect again.


Re needing to say ssl = yes, I thought that was implied for imaps?

I can go back to stunnel, just thought it was an unnecessary layer.

Thanks,

Patrick


On Mon, Jan 21, 2019 at 8:46 PM @lbutlr > wrote:


    On 21 Jan 2019, at 20:17, Patrick Mahan mailto:plma...@gmail.com>> wrote:
 > Due to comcast buisness ISP intercepting imaps

    At you sure about that? I've been using comcast business for 7 years
    and the do not block 143, 993 587 or 25. they do block 110, but
    that's fine, I stopped supporting POP around 2001.

    Other than 110, they block DHCP, NETBIOS, SNMP, and ports 445, 520,
    and 1080. They will block port 25 on a individual basis, but I've no
    idea what their criteria is for that.

 > I need to have my clients connect to non-standard port (). 
    Previously I had been using stunnel to receive the imaps connection

    and forward it to the imap port over 127.0.0.1.  But I would like to
    retire stunnel and have my imap clients connect remotely.

    An stunnel or a reverse proxy is the best way to do this, honestly.

    As for why your config isn't working, my only guess is maybe you
    need to specify ssl?

  inet_listener imaps {
       port = 999
       ssl = yes
    }

    ?


    --     If you write the word "monkey" a million times, do you 
start to

    think you're
    Shakespeare? -- Steven Wright

Re: Changing the imaps port #

[Michael A. Peters]

Comcast DNS servers enforce dnssec, AT does not (last I checked). If 
by chance your zone has DNSSEC enabled but mis-configured then it is 
possible the domain name you use for the dovecot server is not resolving 
because of a dnssec validation failure.


I have never heard of comcast or any ISP blocking port 993. That would 
seem to be a violation of net neutrality rules. I use comcast (consumer, 
not business) and it does not block 993 (does block 25 but that it 
should block for dynamic issued addresses)


Look at the domain name used in your e-mail client and make sure it 
actually resolves. If it does not, check to see if DNSSEC validation is 
the issue.


On 1/21/19 8:58 PM, Patrick Mahan wrote:
Yes, I am pretty sure about that.  I originally was connected via AT 
DSL but wanted the fast access of cable modem.  I need permanent IPs 
which required me to contract with Comcast buisness.  Once I switched 
over, I was no longer able to access my imap server, which was as I 
mentioned, stunnel listening on the imaps port and forwarding to dovecot 
listening on the imap port.


I was getting connection refused on my laptop (thunderbird) email client 
when I was not at home.  I validated that it was not because it was 
reaching my email server.  So who ever was rejecting it, I assumed it 
was somewhere inside the comcast network.  Once I switch to a 
non-standard port, I was able to connect again.


Re needing to say ssl = yes, I thought that was implied for imaps?

I can go back to stunnel, just thought it was an unnecessary layer.

Thanks,

Patrick


On Mon, Jan 21, 2019 at 8:46 PM @lbutlr > wrote:


On 21 Jan 2019, at 20:17, Patrick Mahan mailto:plma...@gmail.com>> wrote:
 > Due to comcast buisness ISP intercepting imaps

At you sure about that? I've been using comcast business for 7 years
and the do not block 143, 993 587 or 25. they do block 110, but
that's fine, I stopped supporting POP around 2001.

Other than 110, they block DHCP, NETBIOS, SNMP, and ports 445, 520,
and 1080. They will block port 25 on a individual basis, but I've no
idea what their criteria is for that.

 > I need to have my clients connect to non-standard port (). 
Previously I had been using stunnel to receive the imaps connection

and forward it to the imap port over 127.0.0.1.  But I would like to
retire stunnel and have my imap clients connect remotely.

An stunnel or a reverse proxy is the best way to do this, honestly.

As for why your config isn't working, my only guess is maybe you
need to specify ssl?

  inet_listener imaps {
       port = 999
       ssl = yes
    }

?


-- 
If you write the word "monkey" a million times, do you start to

think you're
Shakespeare? -- Steven Wright

Re: Solr -> Xapian ?

[Joan Moreau via dovecot]

To get back on thi "build_more" function: 

this is what I receive: 

(see below) 


2 poitns : the header name seems to be added at the end of the *data.
not always, why so ? 

where is the body ? 


Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(mime-version)=1.0MIME-VERSION
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(mime-version)=1.0MIME-VERSION
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
Start indexing 'Sent'
(/data/mail/grosjo.net/jom/xapian-indexes/db_49fdf110ec9bc14c375bd6a3092d)
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(content-type)=MULTIPART/ALTERNATIVE;
BOUNDARY="=_87A48D791CC8B262204294719234352F"CONTENT-TYPE
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(content-type)=MULTIPART/ALTERNATIVE;
BOUNDARY="=_87A48D791CC8B262204294719234352F"CONTENT-TYPE
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(date)=TUE, 22 JAN 2019 09:25:49 +0100DATE
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(date)=TUE, 22 JAN 2019 09:25:49 +0100DATE
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(from)="JOAN MOREAU" 
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(from)="JOAN MOREAU" 
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(to)="JOAN MOREAU" 
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(to)="JOAN MOREAU" 
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(subject)=TESTSUBJECT
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(subject)=TESTSUBJECT
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(user-agent)=ROUNDCUBE WEBMAIL/1.4-GITUSER-AGENT
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(user-agent)=ROUNDCUBE WEBMAIL/1.4-GITUSER-AGENT
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(message-id)=<1c18523a5a00849c8be7970f44276...@grosjo.net>MESSAGE-ID
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(message-id)=<1c18523a5a00849c8be7970f44276...@grosjo.net>MESSAGE-ID
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(x-sender)=j...@grosjo.netx-SENDER
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(x-sender)=j...@grosjo.netx-SENDER
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(content-transfer-encoding)=7BITCONTENT-TRANSFER-ENCODING
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(content-transfer-encoding)=7BITCONTENT-TRANSFER-ENCODING
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(content-type)=TEXT/PLAIN; CHARSET=UTF-8; FORMAT=FLOWEDCONTENT-TYPE
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(content-type)=TEXT/PLAIN; CHARSET=UTF-8; FORMAT=FLOWEDCONTENT-TYPE
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(content-transfer-encoding)=QUOTED-PRINTABLECONTENT-TRANSFER-ENCODING
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(content-transfer-encoding)=QUOTED-PRINTABLECONTENT-TRANSFER-ENCODING
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA(content-type)=TEXT/HTML; CHARSET=UTF-8CONTENT-TYPE
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
DATA2(content-type)=TEXT/HTML; CHARSET=UTF-8CONTENT-TYPE
Jan 22 08:25:50 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
Done indexing 'Sent' (1 msgs in 3 ms, rate: 333.3)
Jan 22 08:25:50 gjserver dovecot[20984]: imap-login: Login:
user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1,
mpid=21699, secured, session=<99GeuQeANlt/AAAB>
Jan 22 08:25:51 gjserver dovecot[20984]:
imap(j...@grosjo.net)<21699><99GeuQeANlt/AAAB>: Logged out in=20201
out=567147 deleted=0 expunged=0 trashed=0 hdr_count=200 hdr_bytes=62139
body_count=0 body_bytes=0
Jan 22 08:25:51 gjserver dovecot[20984]:
indexer-worker(j...@grosjo.net)<20998>:
Indexed 1 messages in Sent (UIDs 60585..60585) 


On 2019-01-08 04:24, Timo Sirainen wrote:

On 7 Jan 2019, at 16.05, Joan Moreau via dovecot  wrote: 


Hi

ANyone to answer specifically ?

Q1 : get_last_uid -> Is this the last UID indexed (which may be not the 
greatest value), or the gratest value (which may not be the latest) (the code of 
existing plugins is unclear about this, Solr looks for the greatest for insance)


All the mails are always supposed to be indexed from the beginning to the last 
indexed mail. If there's a gap,