Re: LMTP Authentication Error

[David Morsberger]

On 2020-10-07 12:43, Alexander Dalloz wrote:
> Am 07.10.2020 um 18:20 schrieb da...@mmpcrofton.com:
>> Any ideas on how to resolve the Userdb connect/lookup problem? My users are 
>> pinging me on Sieve support.
>> Thanks,
>> David
> Provide a full output of "doveconf -n"?
> Alexander

Alexandar,

Thanks and here you go.

# 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.7.2 ()
# OS: Linux 5.4.0-48-generic x86_64 Ubuntu 20.04.1 LTS
# Hostname: mmp-mail.mmpcrofton.com
base_dir = /var/run/dovecot/
first_valid_uid = 150
login_greeting = Dovecot ready.
mail_gid = 150
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_privileged_group = mail
mail_uid = 150
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date index ihave duplicate 
mime foreverypart extracttext
namespace inbox {
 inbox = yes
 location =
 mailbox Drafts {
   auto = subscribe
   special_use = \Drafts
 }
 mailbox Junk {
   auto = subscribe
   special_use = \Junk
 }
 mailbox Sent {
   auto = subscribe
   special_use = \Sent
 }
 mailbox "Sent Messages" {
   auto = no
   special_use = \Sent
 }
 mailbox Spam {
   auto = create
   special_use = \Junk
 }
 mailbox Trash {
   auto = subscribe
   special_use = \Trash
 }
 prefix =
}
passdb {
 args = /etc/dovecot/dovecot-sql.conf.ext
 driver = sql
}
plugin {
 sieve = file:/home/mail/rules/%u/;active=/home/mail/rules/%u/.dovecot.sieve
 sieve_dir = /home/mail/rules/%u
}
protocols = " imap lmtp sieve pop3 sieve"
service auth {
 unix_listener /var/spool/postfix/private/auth {
   group = postfix
   mode = 0660
   user = postfix
 }
}
service lmtp {
 unix_listener /var/spool/postfix/private/dovecot-lmtp {
   group = postfix
   mode = 0600
   user = postfix
 }
}
ssl = required
ssl_cert = 

Re: Version controlled (git) Maildir generated by Dovecot

[Hendrik Boom]

On Wed, Oct 07, 2020 at 02:58:41PM +, Bernd Petrovitsch wrote:
> Hi all!
> 
> On 07/10/2020 14:45, David Myers wrote:
> [...] 
> > Just for completions sake, and in case someone else comes here in the 
> > future;
> 
> +1!
> 
> > This is a link to the current (2020 LO 7) wiki page describing the XML 
> > format 
> > .
> > 
> > However it doesn't mention about being able to save a document directly as 
> > its constituent XML docs ??? so maybe the function has been removed, I have 
> > miss remembered, or I am going mad (I vote for the 4th option ;) ).
> 
> In e.g. localc, you can "save as" an select at the bottom
> right "Flat XML ODF Spreadsheet (.fods)".
> 
> The same exists in lowriter (and very probably in all
> others - I didn't use them up to now).

In LibreOffice you can save a text document as .fodt , where you 
can see all the xml.  It does not contain line breaks, so merging 
with most revision control systems won't be useful.

-- hendrik

RE: Setting up Development Environment for Plugins development/ Debugging

[Mrinal Sharma]

Reaching out to anyone who has information on this request?

From: dovecot  On Behalf Of Mrinal Sharma
Sent: Tuesday, October 6, 2020 10:04 AM
To: dovecot@dovecot.org
Subject: Setting up Development Environment for Plugins development/ Debugging

CAUTION - EXTERNAL EMAIL This email originated from outside of Smith Micro 
Software. Do not click links or open attachments unless you recognize the 
sender and know the content is safe.

Hello Everyone,

I am very new to Dovecot development so I am sorry for asking this very basic 
question. Is there a document which can be referred to setup/debug plugin 
development environment?

Thanks,
Mrinal Sharma

Re: LMTP Authentication Error

[Alexander Dalloz]

Am 07.10.2020 um 18:20 schrieb da...@mmpcrofton.com:
Any ideas on how to resolve the Userdb connect/lookup problem? My users 
are pinging me on Sieve support.


Thanks,
David


Provide a full output of "doveconf -n"?

Alexander

Re: Version controlled (git) Maildir generated by Dovecot

[Gerald Galster]

>> -- Původní e-mail --
>> Od: Gerald Galster 
>> Komu: dovecot@dovecot.org
>> Datum: 7. 10. 2020 14:36:43
>> Předmět: Re: Version controlled (git) Maildir generated by Dovecot
>> 
>>> Could you please tell me / do you know if those dovecot* files have to be 
>>> also backed / archived?
>> 
>> IIRC the plain files from cur/ and new/ should suffice.
>> 
>> You can try it: create a new IMAP folder, then close your mail app.
>> Copy some mail files from another cur/ to this imap folder's cur/ directory.
>> Reopen your mail app and see if new mails are fetched.


> Or on a testing system I can just stop Dovecot, remove everything that 
> doesn't start with a number and restart Dovecot and connect Thunderbird and 
> see the results. Thank you.

This depends on what "everything" means.

I suggested to create a new imap folder because

- the Maildir is clean (necessary folders, correct permissions, no emails, ...)

- neither the server nor the client (thunderbird) did cache anything yet
  (no cache-invalidation, no re-download of old mails, ...)

- the client (thunderbird) created and autosubscribed the folder
  (subscribed folders are also stored on the server, like acls, ...)

With your approach you may (or may not) test more than just restoring a few 
mails inside an empty Maildir.

Best regards
Gerald

Re: LMTP Authentication Error

[david]

Any ideas on how to resolve the Userdb connect/lookup problem? My users 
are pinging me on Sieve support.


Thanks,
David

On 2020-10-04 14:59, David Morsberger wrote:

I am stuck enabling LMTP so that I can use Sieve rules. I have a
postfix -> dovecot setup on Ubuntu. I am using virtual users with a
SQL backend for virtual_users, password maps, and maildir.

I have searched for a solution and implemented everything I have read.


I do not understand why lmtp is trying to go to
/var/run/dovecot/auth-userdb (file does not exist). Everything works
until I uncomment "virtual_transport =
lmtp:unix:private/dovecot-lmtp” in postfix/main.cf

Anyone have a solution?

Here is the lmtp error with surrounding messages

Oct  4 00:37:29 mmp-mail dovecot:
lmtp(i...@domain.com)<268290>: Debug:
auth-master: userdb lookup(i...@domain.com): Started userdb lookup
Oct  4 00:37:29 mmp-mail dovecot:
lmtp(i...@domain.com)<268290>: Debug:
auth-master: conn unix:/var/run/dovecot//auth-userdb: Connecting
Oct  4 00:37:29 mmp-mail dovecot:
lmtp(i...@domain.com)<268290>: Error:
auth-master: userdb lookup(i...@domain.com):
connect(/var/run/dovecot//auth-userdb) failed: No such file or
directory
Oct  4 00:37:29 mmp-mail dovecot:
lmtp(i...@domain.com)<268290>: Debug:
auth-master: userdb lookup(i...@domain.com): Userdb lookup failed
Oct  4 00:37:29 mmp-mail dovecot: lmtp(268290): Error: lmtp-server:
conn unix:pid=268289,uid=129 [1]: rcpt i...@domain.com: Failed to
lookup user i...@domain.com: Internal error occurred. Refer to server
log for more information.

I do see "auth_socket_path = auth-userdb” in doveconf output

In 10-master.conf (-rw-r--r-- 1 root root  3784 Oct  4 00:26
10-master.conf)

service auth {
  unix_listener auth-userdb {
path = /var/spool/postfix/private/auth
mode = 0660
user = postfix
group = postfix
  }
}

In auth-sql-conf.ext (-rw-r--r-- 1 root root   785 Oct  3 22:39
auth-sql.conf.ext)

userdb {
  driver = sql
  args = /etc/dovecot/dovecot-sql.conf.ext
}

In dovecot-sql.conf.ext (-rw-r- 1 root dovecot 6086 Oct  3 22:37
dovecot-sql.conf.ext)

user_query = \
  SELECT email as user, 150 AS uid, 150 AS gid
'maildir:/home/mail/'||maildir as mail \
  FROM virtual_mailbox_maps WHERE email = '%u'

Re: Version controlled (git) Maildir generated by Dovecot

[Bernd Petrovitsch]

Hi all!

On 07/10/2020 14:45, David Myers wrote:
[...] 
> Just for completions sake, and in case someone else comes here in the future;

+1!

> This is a link to the current (2020 LO 7) wiki page describing the XML format 
> .
> 
> However it doesn't mention about being able to save a document directly as 
> its constituent XML docs ??? so maybe the function has been removed, I have 
> miss remembered, or I am going mad (I vote for the 4th option ;) ).

In e.g. localc, you can "save as" an select at the bottom
right "Flat XML ODF Spreadsheet (.fods)".

The same exists in lowriter (and very probably in all
others - I didn't use them up to now).

Mfg,
Bernd
-- 
There is no cloud, just other people computers.
-- https://static.fsf.org/nosvn/stickers/thereisnocloud.svg


pEpkey.asc
Description: application/pgp-keys

Re: Version controlled (git) Maildir generated by Dovecot

[David Myers]

Adam,

Just for completions sake, and in case someone else comes here in the
future;

This is a link to the current (2020 LO 7) wiki page describing the XML
format

.

However it doesn't mention about being able to save a document directly as
its constituent XML docs ??? so maybe the function has been removed, I have
miss remembered, or I am going mad (I vote for the 4th option ;) ).

David



On Wed, Oct 7, 2020 at 12:31 PM Adam  wrote:

> Hi David,
>
> I've never heard about such feature in LibreOffice. Thank you for letting
> me know.
>
> I don't really see myself using the feature since I'd have to remember it.
> I'm used to store all sort of stuff / binary files in git. My rule of thumb
> is that if the file is bellow 10M, just add/commit it. Is it proper way of
> using git? No. Does it work? Yes ;-).
>
> I think that the difference between us is that I'm used to use git for
> everything and you aren't which means neither you or I are correct ;-).
>
> Kind regards,
> Adam
>
> -- Původní e-mail --
> Od: David Myers 
> Komu: Adam 
> Datum: 7. 10. 2020 10:05:28
> Předmět: Re: Version controlled (git) Maildir generated by Dovecot
> > Hello Adam,
> >
> > thanks for the reply. Sounds fair enough to me. I hadn't thought about
> that last benefit of git. I haven't deleted anything off my pc for years
> ... still got HDD from 15 years ago with 'something' on them ?
> >
> > Sorry this is going to go off topic somewhat  (ok, I've just read it
> again... its gone off topic a lot... again, apologies for that)
> >
> >
> > One more question, if you are using exclusively LibreOffice, I
> understand it
> > has a mode where it will separate the file into its constituent flat,
> text XML
> > files (style, contents, formatting etc), all of which can then be stored
> in git
> > with all the advantages that privides, no binary files needed. Do you
> use this
> > functionality ? I haven't done this so I don't know how it impacts the
> work flow
> > for a user, or how it will integrate into a git workflow, but would be
> > interested to hear a user experience. I just use the inbuilt
> 'versioning' that
> > is available within libreoffice (much better than multiple copies of the
> same
> > file with just a few changes).
> >
> > Hopefully my last set of 'novice questions' ;)
> >
> > thanks in advance.
> >
> > David
> >
> > On Wed, Oct 7, 2020 at 10:41 AM Adam  wrote:
> > Hi David,
> >
> > Please find answers bellow.
> >
> > Kind regards,
> > Adam
> >
> > -- Původní e-mail --
> > Od: David Myers 
> > Komu: Adam , Dovecot Mailing List <
> dovecot@dovecot.org>
> > Datum: 7. 10. 2020 8:44:28
> > Předmět: Re: Version controlled (git) Maildir generated by Dovecot
> > Hello Adam, and the dovecot list
> >
> > > Just a question, I hate to pollute the thread, so feel free to push
> these
> > > questions into a new thread if deemed necessary. So as you can guess
> I'm a bit
> > > of a newb here, so rather obvious questions are about to arrive
> > >
> > > As you are using GIT for your archive (which is a cool idea by the
> way) I'm
> > > sure you are well aware that not all files types play nicely with
> version
> > > control, my question therefore is : How do you plan to handle
> attachments ?
> >
> > I use git for everything including for example LibreOffice / Word
> documents. Git works just fine with binary files. You can't use text tools
> like "git diff" but... it works.
> >
> > > Also, although I appreciate the idea of using git, emails generally
> don't
> > > 'change', but I guess that also depends on how you are storing them
> (single
> > > email with links to previous / next ... etc, or as a single big file
> for each
> > > specific thread). Although this is hitting my limits of understanding
> for how
> > > dovecot works, so I probably need educating on this (a pointer to the
> docs would
> > > be good).
> >
> > As I mentioned in the first e-mail, I configured Dovecot to use Maildir
> format -> each e-mail is a single text file. Mail body + attachment(s) are
> in the same file, attachment(s) are Base64 encoded.
> >
> > > You seem concerned regarding the files that you are ignoring that you
> will need
> > > to 'recreate them', so why not do a complete git add . prior to adding
> them into
> > > the git ignore, then you have an initial state for those files too.
> >
> > But I don't want to store files that can be regenerated. I don't want to
> backup stuff, that doesn't have information value.
> >
> > > Final thought, what advantage do you envisage by using git as opposed
> to simply
> > > using a filter to select the files over a certain age, and place them
> into a
> > > zipped TAR archive ? Although I guess you could eventually zip the git
> archive
> > > too, and in the interim it would remain searchable by your users mail
> clients
> > > whilt in git.
> >
> > I like to use git ;-). Tar will 

Re: Version controlled (git) Maildir generated by Dovecot

[Adam]

Or on a testing system I can just stop Dovecot, remove everything that doesn't 
start with a number and restart Dovecot and connect Thunderbird and see the 
results. Thank you.

> -- Původní e-mail --
> Od: Gerald Galster 
> Komu: dovecot@dovecot.org
> Datum: 7. 10. 2020 14:36:43
> Předmět: Re: Version controlled (git) Maildir generated by Dovecot
>
> > Could you please tell me / do you know if those dovecot* files have to be 
> > also backed / archived?
>
> IIRC the plain files from cur/ and new/ should suffice.
>
> You can try it: create a new IMAP folder, then close your mail app.
> Copy some mail files from another cur/ to this imap folder's cur/ directory.
> Reopen your mail app and see if new mails are fetched.
>
> Best regards
> Gerald

Re: Providers running dovecot?

[Olivier Cailloux]

Le dimanche 27 septembre 2020 à 16:30 +0200, Olivier Cailloux a écrit :
> Dear list,
> 
> I am looking for providers of free e-mail addresses known to run
> Dovecot (or a variant thereof) for IMAP access. I need only a few MB
> storage space and no particular features beyond SMTP and IMAP.
> 
> The reason I ask is that Dovecot is known to implement the IMAP spec
> quite respectfully, and I am writing a software which uses IMAP
> search
> (so I would suggest my users to register an e-mail to a provider
> implementing correctly the IMAP Search specifications, to reduce the
> probability of bugs). (More details here: 
> https://www.webhostingtalk.com/showthread.php?t=1821627
> .)
> 
> Do you know of any such provider?

Thanks to those who responded, offering me access to their servers. (I
am sorry, it seems that I can’t reply to these answers individually
when they are not in my inbox; something I had not realized initially,
having not used mailing lists since long.)

These private offers are very kind, but my question was more about
finding a provider who offers this access as a normal service, not as
some special favor to me. That’s because I want to recommend this
provider to the users of a software I am developing. And I do not
expect my users will agree to pay some fee (even a low fee) to register
for an e-mail address just to use my software, so I’d recommend only a
provider who gives starter plans for free. (Of course these users in
turn would perhaps then upgrade their plan if they want to.)

So far I didn’t find a service provider providing free e-mail accounts
(similar to GMail, Yahoo, …) and using Dovecot, which I find very
surprising, as I thought some of these big names, or at least some
smaller ones that I do not know, would use Dovecot.

Thanks again for the replies, anyway.
-- 
Olivier

Re: Version controlled (git) Maildir generated by Dovecot

[Gerald Galster]

> Could you please tell me / do you know if those dovecot* files have to be 
> also backed / archived?

IIRC the plain files from cur/ and new/ should suffice.

You can try it: create a new IMAP folder, then close your mail app.
Copy some mail files from another cur/ to this imap folder's cur/ directory.
Reopen your mail app and see if new mails are fetched.

Best regards
Gerald

Re: Version controlled (git) Maildir generated by Dovecot

[Robert Schetterer]

Am 07.10.20 um 13:20 schrieb Adam:

Thank you Vitalii.

Could you please tell me / do you know if those dovecot* files have to be also 
backed / archived?

Kind regards,

Adam


-- Původní e-mail --
Od: Vitalii 
Komu: Adam 
Datum: 7. 10. 2020 12:04:11
Předmět: Re: Version controlled (git) Maildir generated by Dovecot
My 5 cents:
Git is REVISION control, It make sense to use it if files are modified.
Email is static files so it's better to use any backup system with
archiving capabilities (gzip or similar). It provide ability to copy
partially or just some old years to another HDD that is not possible
with git.
Also you can archive by dsync to that new server with account on mdbox
account with gzip compression. That will save space and gives you full
access to all email.

P.S. there are many options to choose. I use borg to backup
everything, including email and mdbox format with separate folder for
attachments that deduplicate attachments on backups and in filesystem.







Hi, with maildir you can just use rsync
for pure backup

for archive i.e

something like

https://blog.sys4.de/mailarchiv-mit-dovecot-und-postfix-sortiert-nach-datum-mailadressen-und-ein-ausgehend-unterordnern-de.html

but there are many ways to goal this

--
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG, 80333 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein

Re: Version controlled (git) Maildir generated by Dovecot

[Adam]

Thank you Vitalii.

Could you please tell me / do you know if those dovecot* files have to be also 
backed / archived?

Kind regards,

Adam

> -- Původní e-mail --
> Od: Vitalii 
> Komu: Adam 
> Datum: 7. 10. 2020 12:04:11
> Předmět: Re: Version controlled (git) Maildir generated by Dovecot
> My 5 cents:
> Git is REVISION control, It make sense to use it if files are modified.
> Email is static files so it's better to use any backup system with
> archiving capabilities (gzip or similar). It provide ability to copy
> partially or just some old years to another HDD that is not possible
> with git.
> Also you can archive by dsync to that new server with account on mdbox
> account with gzip compression. That will save space and gives you full
> access to all email.
>
> P.S. there are many options to choose. I use borg to backup
> everything, including email and mdbox format with separate folder for
> attachments that deduplicate attachments on backups and in filesystem.

Re: Version controlled (git) Maildir generated by Dovecot

[Vitalii]

My 5 cents:
Git is REVISION control, It make sense to use it if files are modified.
Email is static files so it's better to use any backup system with
archiving capabilities (gzip or similar). It provide ability to copy
partially or just some old years to another HDD that is not possible
with git.
Also you can archive by dsync to that new server with account on mdbox
account with gzip compression. That will save space and gives you full
access to all email.

P.S. there are many options to choose. I use borg to backup
everything, including email and mdbox format with separate folder for
attachments that deduplicate attachments on backups and in filesystem.


On Wed, 07 Oct 2020 12:31:44 +0200 (CEST)
Adam  wrote:

Re: Version controlled (git) Maildir generated by Dovecot

[Adam]

Hi David,

I've never heard about such feature in LibreOffice. Thank you for letting me 
know.

I don't really see myself using the feature since I'd have to remember it. I'm 
used to store all sort of stuff / binary files in git. My rule of thumb is that 
if the file is bellow 10M, just add/commit it. Is it proper way of using git? 
No. Does it work? Yes ;-).

I think that the difference between us is that I'm used to use git for 
everything and you aren't which means neither you or I are correct ;-).

Kind regards,
Adam

-- Původní e-mail --
Od: David Myers 
Komu: Adam 
Datum: 7. 10. 2020 10:05:28
Předmět: Re: Version controlled (git) Maildir generated by Dovecot
> Hello Adam,
>
> thanks for the reply. Sounds fair enough to me. I hadn't thought about that 
> last benefit of git. I haven't deleted anything off my pc for years ... still 
> got HDD from 15 years ago with 'something' on them ?
>
> Sorry this is going to go off topic somewhat  (ok, I've just read it 
> again... its gone off topic a lot... again, apologies for that)
>
>
> One more question, if you are using exclusively LibreOffice, I understand it
> has a mode where it will separate the file into its constituent flat, text XML
> files (style, contents, formatting etc), all of which can then be stored in 
> git
> with all the advantages that privides, no binary files needed. Do you use this
> functionality ? I haven't done this so I don't know how it impacts the work 
> flow
> for a user, or how it will integrate into a git workflow, but would be
> interested to hear a user experience. I just use the inbuilt 'versioning' that
> is available within libreoffice (much better than multiple copies of the same
> file with just a few changes).
>
> Hopefully my last set of 'novice questions' ;)
>
> thanks in advance.
>
> David
>
> On Wed, Oct 7, 2020 at 10:41 AM Adam  wrote:
> Hi David,
>
> Please find answers bellow.
>
> Kind regards,
> Adam
>
> -- Původní e-mail --
> Od: David Myers 
> Komu: Adam , Dovecot Mailing List 
> Datum: 7. 10. 2020 8:44:28
> Předmět: Re: Version controlled (git) Maildir generated by Dovecot
> Hello Adam, and the dovecot list
>
> > Just a question, I hate to pollute the thread, so feel free to push these
> > questions into a new thread if deemed necessary. So as you can guess I'm a 
> > bit
> > of a newb here, so rather obvious questions are about to arrive
> >
> > As you are using GIT for your archive (which is a cool idea by the way) I'm
> > sure you are well aware that not all files types play nicely with version
> > control, my question therefore is : How do you plan to handle attachments ?
>
> I use git for everything including for example LibreOffice / Word documents. 
> Git works just fine with binary files. You can't use text tools like "git 
> diff" but... it works.
>
> > Also, although I appreciate the idea of using git, emails generally don't
> > 'change', but I guess that also depends on how you are storing them (single
> > email with links to previous / next ... etc, or as a single big file for 
> > each
> > specific thread). Although this is hitting my limits of understanding for 
> > how
> > dovecot works, so I probably need educating on this (a pointer to the docs 
> > would
> > be good).
>
> As I mentioned in the first e-mail, I configured Dovecot to use Maildir 
> format -> each e-mail is a single text file. Mail body + attachment(s) are in 
> the same file, attachment(s) are Base64 encoded.
>
> > You seem concerned regarding the files that you are ignoring that you will 
> > need
> > to 'recreate them', so why not do a complete git add . prior to adding them 
> > into
> > the git ignore, then you have an initial state for those files too.
>
> But I don't want to store files that can be regenerated. I don't want to 
> backup stuff, that doesn't have information value.
>
> > Final thought, what advantage do you envisage by using git as opposed to 
> > simply
> > using a filter to select the files over a certain age, and place them into a
> > zipped TAR archive ? Although I guess you could eventually zip the git 
> > archive
> > too, and in the interim it would remain searchable by your users mail 
> > clients
> > whilt in git.
>
> I like to use git ;-). Tar will work just fine.
>
> In this use case the only real benefit of git is that it never forgets. 
> Unless I delete whole .git directory, I can make a mistake, delete some 
> e-mails (files), commit changes and rollback. I can't rollback if I delete 
> tar archive.
>
> > Thanks in advance, and apologies once again for polluting your question 
> > with my
> > own.

Re: Version controlled (git) Maildir generated by Dovecot

[David Myers]

Hello Adam,

thanks for the reply. Sounds fair enough to me. I hadn't thought about that
last benefit of git. I haven't deleted anything off my pc for years ...
still got HDD from 15 years ago with 'something' on them ?

Sorry this is going to go off topic somewhat  (ok, I've just read it
again... its gone off topic a lot... again, apologies for that)

One more question, if you are using exclusively LibreOffice, I understand
it has a mode where it will separate the file into its constituent flat,
text XML files (style, contents, formatting etc), all of which can then be
stored in git with all the advantages that privides, no binary files
needed. Do you use this functionality ? I haven't done this so I don't know
how it impacts the work flow for a user, or how it will integrate into a
git workflow, but would be interested to hear a user experience. I just use
the inbuilt 'versioning' that is available within libreoffice (much better
than multiple copies of the same file with just a few changes).

Hopefully my last set of 'novice questions' ;)

thanks in advance.

David

On Wed, Oct 7, 2020 at 10:41 AM Adam  wrote:

> Hi David,
>
> Please find answers bellow.
>
> Kind regards,
> Adam
>
> -- Původní e-mail --
> Od: David Myers 
> Komu: Adam , Dovecot Mailing List <
> dovecot@dovecot.org>
> Datum: 7. 10. 2020 8:44:28
> Předmět: Re: Version controlled (git) Maildir generated by Dovecot
> Hello Adam, and the dovecot list
>
> > Just a question, I hate to pollute the thread, so feel free to push these
> > questions into a new thread if deemed necessary. So as you can guess I'm
> a bit
> > of a newb here, so rather obvious questions are about to arrive
> >
> > As you are using GIT for your archive (which is a cool idea by the way)
> I'm
> > sure you are well aware that not all files types play nicely with version
> > control, my question therefore is : How do you plan to handle
> attachments ?
>
> I use git for everything including for example LibreOffice / Word
> documents. Git works just fine with binary files. You can't use text tools
> like "git diff" but... it works.
>
> > Also, although I appreciate the idea of using git, emails generally don't
> > 'change', but I guess that also depends on how you are storing them
> (single
> > email with links to previous / next ... etc, or as a single big file for
> each
> > specific thread). Although this is hitting my limits of understanding
> for how
> > dovecot works, so I probably need educating on this (a pointer to the
> docs would
> > be good).
>
> As I mentioned in the first e-mail, I configured Dovecot to use Maildir
> format -> each e-mail is a single text file. Mail body + attachment(s) are
> in the same file, attachment(s) are Base64 encoded.
>
> > You seem concerned regarding the files that you are ignoring that you
> will need
> > to 'recreate them', so why not do a complete git add . prior to adding
> them into
> > the git ignore, then you have an initial state for those files too.
>
> But I don't want to store files that can be regenerated. I don't want to
> backup stuff, that doesn't have information value.
>
> > Final thought, what advantage do you envisage by using git as opposed to
> simply
> > using a filter to select the files over a certain age, and place them
> into a
> > zipped TAR archive ? Although I guess you could eventually zip the git
> archive
> > too, and in the interim it would remain searchable by your users mail
> clients
> > whilt in git.
>
> I like to use git ;-). Tar will work just fine.
>
> In this use case the only real benefit of git is that it never forgets.
> Unless I delete whole .git directory, I can make a mistake, delete some
> e-mails (files), commit changes and rollback. I can't rollback if I delete
> tar archive.
>
> > Thanks in advance, and apologies once again for polluting your question
> with my
> > own.
>
> David
>

Re: Version controlled (git) Maildir generated by Dovecot

[Adam]

Hi David,

Please find answers bellow.

Kind regards,
Adam

-- Původní e-mail --
Od: David Myers 
Komu: Adam , Dovecot Mailing List 
Datum: 7. 10. 2020 8:44:28
Předmět: Re: Version controlled (git) Maildir generated by Dovecot
Hello Adam, and the dovecot list

> Just a question, I hate to pollute the thread, so feel free to push these
> questions into a new thread if deemed necessary. So as you can guess I'm a bit
> of a newb here, so rather obvious questions are about to arrive
>
> As you are using GIT for your archive (which is a cool idea by the way) I'm
> sure you are well aware that not all files types play nicely with version
> control, my question therefore is : How do you plan to handle attachments ?

I use git for everything including for example LibreOffice / Word documents. 
Git works just fine with binary files. You can't use text tools like "git diff" 
but... it works.

> Also, although I appreciate the idea of using git, emails generally don't
> 'change', but I guess that also depends on how you are storing them (single
> email with links to previous / next ... etc, or as a single big file for each
> specific thread). Although this is hitting my limits of understanding for how
> dovecot works, so I probably need educating on this (a pointer to the docs 
> would
> be good).

As I mentioned in the first e-mail, I configured Dovecot to use Maildir format 
-> each e-mail is a single text file. Mail body + attachment(s) are in the same 
file, attachment(s) are Base64 encoded.

> You seem concerned regarding the files that you are ignoring that you will 
> need
> to 'recreate them', so why not do a complete git add . prior to adding them 
> into
> the git ignore, then you have an initial state for those files too.

But I don't want to store files that can be regenerated. I don't want to backup 
stuff, that doesn't have information value.

> Final thought, what advantage do you envisage by using git as opposed to 
> simply
> using a filter to select the files over a certain age, and place them into a
> zipped TAR archive ? Although I guess you could eventually zip the git archive
> too, and in the interim it would remain searchable by your users mail clients
> whilt in git.

I like to use git ;-). Tar will work just fine.

In this use case the only real benefit of git is that it never forgets. Unless 
I delete whole .git directory, I can make a mistake, delete some e-mails 
(files), commit changes and rollback. I can't rollback if I delete tar archive.

> Thanks in advance, and apologies once again for polluting your question with 
> my
> own.

David

Re: Version controlled (git) Maildir generated by Dovecot

[David Myers]

Hello Adam, and the dovecot list

Just a question, I hate to pollute the thread, so feel free to push these
questions into a new thread if deemed necessary. So as you can guess I'm a
bit of a newb here, so rather obvious questions are about to arrive

As you are using GIT for your archive (which is a cool idea by the way) I'm
sure you are well aware that not all files types play nicely with version
control, my question therefore is : How do you plan to handle attachments ?

Also, although I appreciate the idea of using git, emails generally don't
'change', but I guess that also depends on how you are storing them (single
email with links to previous / next ... etc, or as a single big file for
each specific thread). Although this is hitting my limits of understanding
for how dovecot works, so I probably need educating on this (a pointer to
the docs would be good).

You seem concerned regarding the files that you are ignoring that you will
need to 'recreate them', so why not do a complete git add . prior to adding
them into the git ignore, then you have an initial state for those files
too.

Final thought, what advantage do you envisage by using git as opposed to
simply using a filter to select the files over a certain age, and place
them into a zipped TAR archive ? Although I guess you could eventually zip
the git archive too, and in the interim it would remain searchable by your
users mail clients whilt in git.

Thanks in advance, and apologies once again for polluting your question
with my own.

David











On Tue, Oct 6, 2020 at 7:22 PM Adam  wrote:

> Hi Everybody,
>
> I'd like to start archiving e-mails by moving them to a server with
> running Dovecot.
>
> I installed "dovecot-core" and "dovecot-imapd" (version 2.3.4.1) on Debian
> 10.
>
> One of a few configurations I made is to use Maildir:
>
> # grep '^mail_location = ' /etc/dovecot/conf.d/10-mail.conf
> mail_location = maildir:~/Maildir
>
> I successfully moved some e-mails (at this moment it's just a test) to
> subfolders that describe a year of the backup.
>
> I'd like to start using git on the whole Maildir. After going through the
> Maildir directory and subdirectories I realized that Dovecot saves some
> extra files that I might not need to backup (version control).
>
> Based on my understanding and after reading
> https://wiki.dovecot.org/IndexFiles I believe that the only files I need
> to archive are files in "cur/" directory and these files should start with
> Unix time.
>
> So I decided to write a ".gitignore" whitelist that will include only
> files in "cur/" that start with a number:
>
> # cat .gitignore
> ---
> !.gitignore
> !*/
> !/Maildir/cur/[0-9]*
> !/Maildir/*/cur/[0-9]*
> ---
>
> and it's working:
>
> # git add . --dry-run
> ---
> add '.gitignore'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M110302P17113.mail1,S=8066,W=8276:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M134554P17113.mail1,S=8024,W=8234:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M157569P17113.mail1,S=8005,W=8215:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M180965P17113.mail1,S=8021,W=8231:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M205518P17113.mail1,S=8017,W=8227:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M355896P17113.mail1,S=8042,W=8252:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M52133P17113.mail1,S=8053,W=8263:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M555847P17113.mail1,S=8029,W=8239:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M581502P17113.mail1,S=8047,W=8257:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M606379P17113.mail1,S=8044,W=8254:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M628850P17113.mail1,S=8036,W=8246:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M657431P17113.mail1,S=8069,W=8279:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M681265P17113.mail1,S=8039,W=8249:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M706585P17113.mail1,S=8073,W=8283:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M730046P17113.mail1,S=8059,W=8269:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M753487P17113.mail1,S=8034,W=8244:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M776202P17113.mail1,S=8019,W=8229:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M799238P17113.mail1,S=8085,W=8295:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M822106P17113.mail1,S=8081,W=8291:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M844397P17113.mail1,S=8102,W=8312:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M867952P17113.mail1,S=7964,W=8174:2,S'
> add
> 'Maildir/.INBOX.years.2020.family/cur/1601986418.M889882P17113.mail1,S=7992,W=8202:2,S'
> add
>