[Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups

[Charles Marcus]

Hi all,

I'm planning on implementing this in my new upcoming dovecot instance, 
and would like to hear thoughts on how best to accomplish this. We will 
be paying Timo's support company to do the work, but obviously, the less 
work in the form of coding he has to do to get this working (I'm hoping 
it won't be a lot), the more money it will save us... ;)

First - I currently use rsnapshot to backup emails, so that is the 
use-case I'm most interested in getting working. It is rsync based, and 
like other rsync based backup programs it uses hardlinks to save storage 
space - so you can have a *lot* of backups (going back months, or even 
years), where each snapshot only adds a little more to the total disk 
space being used.

The snapshots are stored with the following filesystem layout:

/path/to/snapshotsdir/hourly.0
...
/path/to/snapshotsdir/hourly.4
/path/to/snapshotsdir/daily.0
...
/path/to/snapshotsdir/daily.7
/path/to/snapshotsdir/weekly.0
...
/path/to/snapshotsdir/weekly.4
/path/to/snapshotsdir/monthly.0
...
/path/to/snapshotsdir/monthly.12
/path/to/snapshotsdir/yearly.0
...
/path/to/snapshotsdir/yearly.5

The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this 
is a bit confusing to people new to rsnapshot), and would *not* be used 
for displaying the mail folders to the users - it is the Date/Time 
stamps of each of the snapshot dirs above that would be used to display 
the folder names under the 'Time Machine' namespace. This is, I imagine, 
the part that will need some actual coding by Timo to get working - 
maybe just some new config variables added to the namespace code for 
mapping the date/time stamps of the directories to user friendly folder 
names in the namespace.

That said, I'd like to design this and have it coded such that it will 
work with almost any type of backup storage that stores snapshots as 
date/time stamped directories like this (there must be others, right?).

Also, it goes without saying that this code will be (if Timo is ok with 
it) part of the core dovecot code going forward, so anyone else will be 
able to benefit from it.

What I'm envisioning is something like this...

1. Define a namespace - for this example we'll call it 'Time Machine'

2. Under this namespace, each user will see their, and *only* their
   snapshots

So, each user would see something like this:

My Mail Account
   Inbox
   Drafts
   Templates
   Sent
   Time Machine (sorted above user created folders if possible)
        -4/3/12, 8:00am (first subfolder)
           Inbox
           Drafts
           etc... (all other folders and sub-folders shown here)
        +4/3/12, 12:00pm (first subfolder)
        etc...
   Other User Folders
   ...

Or even better, I'm thinking some magical code that can group them by 
Date, like:

        -4/3/12 (first subfolder)
           -8:00am (next sub-folder)
              Inbox
              Drafts
              Etc... (all folders and sub-folders shown here)
           +12:00pm
           +4:00pm
           +8:00pm
        +4/4/12
        etc...

Comments? Suggestions? Flames?

--

Best regards,

Charles