Re: Only some of my sent emails are stored in sent-folder

[Alex via dovecot]

On Thu, Jun 20, 2024 at 10:56:48PM +0200, Nils via dovecot wrote:
> On 6/20/24 22:47, Marc wrote:
> > using outlook?
> > 
> > I also had this, and when I searched for this, I found lots of people have 
> > had this and it is a known issue since many years. (They recommend creating 
> > new profile)
> > 
> Thank you very much for your fast response, Marc.
> 
> I use Thunderbird (currently 115.12.0). I was suspicious of my client too,
> but I manage accounts on other servers with that client too and they seem to
> work without problems (although I don't use aliases there). I'm gonna
> prepare another client and check that.

For Thunderbird I suggest going to "Copies & Folders" of the account and 
replacing
"Place a copy in" with "Bcc these email addresses" option.
Then if you want outgoing mails to be in the Sent folder on IMAP server,
move them there with a Sieve filter.

Storing sent mails over IMAP is unreliable because it is not the same connection
as you use for SMTP. It is always possible that you manage to send the mail out
but fail to connect to IMAP or disconnect from the network before the message
is fully stored there.

The default of storing messages over IMAP looks to me like a historical solution
that evolved by users switching from using TUI clients that stored mails
in the Sent folder using "Fcc" mechanism on the remote *nix machines
to accessing these same *nix machines over IMAP sometimes.
In this case it makes sense to store mails into the same folder.
But now that email servers are accessed exclusively over IMAP
and not remote shell, Bcc to self looks like a better solution.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

SAVEDATE is using ctime instead of btime for Maildir

[Alex via dovecot]

Dovecot supports SAVEDATE IMAP extension [1].
This is however not implemented correctly for Maildir storage.

Compared to "internal date", "save date" is supposed to be the same
but well-defined for APPEND and preserved when COPY command is used.

However, for Maildir storage INTERNALDATE is implemented
using mtime (modification timestamp)
and SAVEDATE is implemented using ctime (status change timestamp) [2].
ctime is changed even more often than mtime.
For example, chown or chmod operations on the message change its
ctime, but preserve its mtime.

Dovecot should instead use btime (creation timestamp) when it is available
and not implement SAVEDATE at all if there is no btime.

Just one example of the message in Maildir:

stat 1708031310.M769290P3712350.nine,S=7898,W=8002:2,Sa
  File: 1708031310.M769290P3712350.nine,S=7898,W=8002:2,Sa
  Size: 7898Blocks: 16 IO Block: 4096   regular file
Device: 9,2 Inode: 7864351 Links: 1
Access: (0600/-rw---)  Uid: (  999/   vmail)   Gid: (  996/   vmail)
Access: 2024-02-16 08:34:04.374097798 +0100
Modify: 2024-02-15 22:08:30.769813368 +0100
Change: 2024-02-21 13:39:14.878163244 +0100
 Birth: 2024-02-15 22:08:30.769813368 +0100

Its Birth date (btime) and Modify date (mtime) are the same, but Change date 
(ctime)
was changed, likely by someone doing chown/chmod manually.

You can read more in the issue [3] where we have problems with incorrect 
INTERNALDATE
and investigated the possibility of using SAVEDATE, but found that it is not 
implemented correctly
and is changed even more often than INTERNALDATE. According to the standard it 
should be the other way round,
SAVEDATE should be more stable and survive COPY.

[1] 
[2] 

[3] 
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Inconsistency in map index with dovecot v2.3.21

[John Alex. via dovecot]

Hi,

since I upgraded dovecot to version 2.3.21 on a FreeBSD 13.2 system, I've been seeing a 
few occurences of the following warnings daily:


Jan 11 13:13:25 dimokritos dovecot[33170]: lmtp(user1)<8209>: 
Warning: mdbox /var/vmail/users/user1/mdbox/storage: Inconsistency in map index (5,40 != 5,56)
Jan 11 13:13:25 dimokritos dovecot[33170]: lmtp(user1)<8209>: 
Warning: fscking index file /var/vmail/users/user1/mdbox/storage/dovecot.map.index
Jan 11 13:13:25 dimokritos dovecot[33170]: lmtp(user1)<8209>: 
Warning: mdbox /var/vmail/users/user1/mdbox/storage: rebuilding indexes
Jan 11 13:13:26 dimokritos dovecot[33170]: lmtp(user1)<8209>: 
Warning: fscking index file /var/vmail/users/user1/mdbox/storage/dovecot.map.index


The numbers in parentheses seem to always be of the format (x,40 != x,56) or 
(x,40 != x,72).

I also get such errors on some users when purging (doveadm-purge fails with 
error 75):

doveadm(user2): Warning: mdbox /var/vmail/users/user2/mdbox/storage: Inconsistency in map 
index (5,40 != 5,72)
doveadm(user2): Warning: fscking index file 
/var/vmail/users/user2/mdbox/storage/dovecot.map.index

doveadm(user2): Warning: mdbox /var/vmail/users/user2/mdbox/storage: rebuilding 
indexes
doveadm(user2): Error: Purging namespace '' failed: BUG: Unknown internal error

If I rerun doveadm purge for this user it completes successfully.

Can anyone shed some light on what might be causing these messages? I didn't have these 
with 2.3.20. Should I be worried about data loss due to these index inconsistencies?

___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Trying to track down source of duplicate messages

[Alex via dovecot]

Hi,

I haven't received any responses to my duplicate messages problem. It
occurred to me that I posted my full dovecot config instead of just
the changes we've made locally. I thought it might help to follow up
with just the specific config to make it easier to identify a
potential problem.

Any ideas greatly appreciated.

# doveconf -n
# 2.3.5 (513208660): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.5 (2483b085)
# OS: Linux 4.20.11-100.fc28.x86_64 x86_64 Fedora release 28 (Twenty Eight)
mail_gid = mail
mail_home = mail/
mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u
mail_max_userip_connections = 500
mail_plugins = " listescape"
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext
mbox_write_locks = fcntl
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix =
}
passdb {
  driver = pam
}
passdb {
  driver = pam
}
protocols = imap pop3
service auth {
  unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
  }
}
service imap-login {
  inet_listener imap {
port = 0
  }
  inet_listener imaps {
port = 993
  }
  process_min_avail = 20
  service_count = 0
}
service pop3-login {
  inet_listener pop3 {
port = 0
  }
  inet_listener pop3s {
address = 192.168.1.100
port = 995
  }
}
ssl_cert =  wrote:
>
> Hi,
>
> I have a fedora28 system with dovecot-2.3.5 and I'm trying to figure
> out why I'm increasingly receiving duplicate emails. The message IDs
> are exactly the same. It's always two of the same messages. I don't
> think it's ever occurred in the inbox. It doesn't happen with every
> message or seemingly in every folder.
>
> I've read about dovecot deduplicate, but of course I want to prevent
> them from being duplicated in the first place. It's also happening
> much more frequently now.
>
> Could something in my configuration cause this problem?
>
> Below is the contents of my doveconf
>

Trying to track down source of duplicate messages

[Alex via dovecot]

Hi,

I have a fedora28 system with dovecot-2.3.5 and I'm trying to figure
out why I'm increasingly receiving duplicate emails. The message IDs
are exactly the same. It's always two of the same messages. I don't
think it's ever occurred in the inbox. It doesn't happen with every
message or seemingly in every folder.

I've read about dovecot deduplicate, but of course I want to prevent
them from being duplicated in the first place. It's also happening
much more frequently now.

Could something in my configuration cause this problem?

Below is the contents of my doveconf

# 2.3.5 (513208660): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.5 (2483b085)
# OS: Linux 4.20.11-100.fc28.x86_64 x86_64 Fedora release 28 (Twenty Eight)
# Hostname: orion.inside.guardiandigital.com
# NOTE: Send doveconf -n output instead when asking for help.
auth_anonymous_username = anonymous
auth_cache_negative_ttl = 1 hours
auth_cache_size = 0
auth_cache_ttl = 1 hours
auth_cache_verify_password_with_worker = no
auth_debug = no
auth_debug_passwords = no
auth_default_realm =
auth_failure_delay = 2 secs
auth_gssapi_hostname =
auth_krb5_keytab =
auth_master_user_separator =
auth_mechanisms = plain
auth_policy_check_after_auth = yes
auth_policy_check_before_auth = yes
auth_policy_hash_mech = sha256
auth_policy_hash_nonce =
auth_policy_hash_truncate = 12
auth_policy_log_only = no
auth_policy_reject_on_fail = no
auth_policy_report_after_auth = yes
auth_policy_request_attributes = login=%{requested_username}
pwhash=%{hashed_password} remote=%{rip} device_id=%{client_id}
protocol=%s
auth_policy_server_api_header =
auth_policy_server_timeout_msecs = 2000
auth_policy_server_url =
auth_proxy_self =
auth_realms =
auth_socket_path = auth-userdb
auth_ssl_require_client_cert = no
auth_ssl_username_from_cert = no
auth_stats = no
auth_use_winbind = no
auth_username_chars =
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@
auth_username_format = %Lu
auth_username_translation =
auth_verbose = no
auth_verbose_passwords = no
auth_winbind_helper_path = /usr/bin/ntlm_auth
auth_worker_max_count = 30
base_dir = /var/run/dovecot
config_cache_size = 1 M
debug_log_path =
default_client_limit = 1000
default_idle_kill = 1 mins
default_internal_group = dovecot
default_internal_user = dovecot
default_login_user = dovenull
default_process_limit = 100
default_vsz_limit = 256 M
deliver_log_format = msgid=%m: %$
dict_db_config =
director_flush_socket =
director_mail_servers =
director_max_parallel_kicks = 100
director_max_parallel_moves = 100
director_output_buffer_size = 10 M
director_ping_idle_timeout = 30 secs
director_ping_max_timeout = 1 mins
director_servers =
director_user_expire = 15 mins
director_user_kick_delay = 2 secs
director_username_hash = %u
disable_plaintext_auth = yes
dotlock_use_excl = yes
doveadm_allowed_commands =
doveadm_api_key =
doveadm_http_rawlog_dir =
doveadm_password =
doveadm_port = 0
doveadm_socket_path = doveadm-server
doveadm_username = doveadm
doveadm_worker_count = 0
dsync_alt_char = _
dsync_commit_msgs_interval = 100
dsync_features =
dsync_hashed_headers = Date Message-ID
dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -U
first_valid_gid = 1
first_valid_uid = 500
haproxy_timeout = 3 secs
haproxy_trusted_networks =
hostname =
imap_capability =
imap_client_workarounds =
imap_fetch_failure = disconnect-immediately
imap_hibernate_timeout = 0
imap_id_log =
imap_id_retain = no
imap_id_send = name *
imap_idle_notify_interval = 2 mins
imap_literal_minus = no
imap_logout_format = in=%i out=%o deleted=%{deleted}
expunged=%{expunged} trashed=%{trashed} hdr_count=%{fetch_hdr_count}
hdr_bytes=%{fetch_hdr_bytes} body_count=%{fetch_body_count}
body_bytes=%{fetch_body_bytes}
imap_max_line_length = 64 k
imap_metadata = no
imap_urlauth_host =
imap_urlauth_logout_format = in=%i out=%o
imap_urlauth_port = 143
imapc_cmd_timeout = 5 mins
imapc_connection_retry_count = 1
imapc_connection_retry_interval = 1 secs
imapc_features =
imapc_host =
imapc_list_prefix =
imapc_master_user =
imapc_max_idle_time = 29 mins
imapc_max_line_length = 0
imapc_password =
imapc_port = 143
imapc_rawlog_dir =
imapc_sasl_mechanisms =
imapc_ssl = no
imapc_ssl_verify = yes
imapc_user =
import_environment = TZ CORE_OUTOFMEM CORE_ERROR LISTEN_PID LISTEN_FDS
info_log_path =
instance_name = dovecot
last_valid_gid = 0
last_valid_uid = 0
lda_mailbox_autocreate = no
lda_mailbox_autosubscribe = no
lda_original_recipient_header =
libexec_dir = /usr/libexec/dovecot
listen = *, ::
lmtp_hdr_delivery_address = final
lmtp_proxy = no
lmtp_proxy_rawlog_dir =
lmtp_rawlog_dir =
lmtp_rcpt_check_quota = no
lmtp_save_to_detail_mailbox = no
lmtp_user_concurrency_limit = 0
lock_method = fcntl
log_core_filter =
log_debug =
log_path = syslog
log_timestamp = "%b %d %H:%M:%S "
login_access_sockets =
login_greeting = Dovecot ready.
login_log_format = %$: %s
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e
%c session=<%{session}>
login_plugin_dir =