Re: Did calculating the quota change from 2.3 to 2.5?

[Adam Tauno Williams via Info-cyrus]

> > If you use imapsync, it doesn't know about that, and will upload
> > the same message twice. 2.5 doesn't have the smarts to recognise
> > that it's the same message.
> imapsync can only sync mail the old server knows about. And in the
> end there is more quota used on the new server!?
> The only explanation is the quota on the old server is broken, isn't
> it?

No, IMAP doesn't know about deduplication;  so imapsync between two
servers dededuplicates.  imapsync may also repair damaged or missing
message headers - meaning the messages are no longer are the same - so
a tool like hardlinks will not return you to the same count in du as on
the old server.

And then there is the [virtuous] issue of delayed expunge.

-- 
Adam Tauno Williams  GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Migrating mailbox data from Cyrus to MicroSoft Office 365 using their import tool.

[Adam Tauno Williams via Info-cyrus]

On Wed, 2016-06-22 at 17:28 +0200, Eric Luyten via Info-cyrus wrote:
> After trying for a couple of days I have come to the conclusion
> that the Office 365 IMAP import tool uses the LOGIN authentication
> mech while Cyrus requires PLAIN or stronger for proxying to work.
> Even when only announcing AUTH=PLAIN in our server capabilities,
> Microsoft executes LOGIN ... ...

Has anyone gotten this to work using an administrative account [cyrus,
for example] and the "optional UserRoot attribute"?  It seems ideally
suited - but it does not appear to work?

EmailAddress,UserName,Password,UserRoot
fred@office365domain,cyrus,P@assw0rd,user.fred

https://support.office.com/en-us/article/CSV-files-for-IMAP-migration-b
atches-187ce085-9a83-4612-80a2-f562b3049fc2


-- 
Adam Tauno Williams  GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Xapian search databases

[Adam Tauno Williams via Info-cyrus]

On Thu, 2016-11-03 at 10:47 +1100, Bron Gondwana via Info-cyrus wrote:
> Just out of interest - is anyone other than Fastmail currently using
> the Cyrus Xapian-based search system?

Not using Xapian.

-- 
Adam Tauno Williams  GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: watching and processing a Spam folder for each user

[Adam Tauno Williams via Info-cyrus]

> While I can see this being a neat built-in feature of a mail server
> like Cyrus IMAP, I doubt it exists.  I'd be happy to be corrected.

Good old fecthmail.

fetchmail --verbose --all --norewrite  \
  --folder 'user.awilliam.SPAM' --mda '/usr/bin/sa-learn --spam'

> I wonder if such a beast exists.  I'd love any pointers if anyone
> knows of such.

Yes, you probably already have it installed.


-- 
Adam Tauno Williams  GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA



signature.asc
Description: This is a digitally signed message part

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: how to deal with mail retention/archival.

[Adam Tauno Williams via Info-cyrus]

On Fri, 2016-08-26 at 12:11 -0500, Jason L Tibbitts III via Info-cyrus
wrote:
> > > > > > "GR(" == Giuseppe Ravasio (LU) via Info-cyrus <
> > > > > > info-cyrus@lists.andrew.cmu.edu> writes:
> GR(> I saw that someone proposed to make a sort of abuse of delayed
> GR(> expunge, but I think that in order to comply with regulatory
> GR(> retention should be better considering some specific software.
> True, but it seems odd (to me, in a situation where I don't have
> infinite money) to have basically two mail servers: one which
> actually
> removes things when the user deletes stuff and one which doesn't.

But that is essentially archival - on-the-same-system is not archival. 
 It is also, potentially, still available to be easily changed;  which
is not good when the intent is retention.

> I guess they can be optimized for different things, but it still 
> seems odd when we already have a server that can store as much mail 
> as you want, provides a means to access and search it with ACLs for 
> auditors and such, and of course is already installed and running.

Do you want to give auditors access to your production systems?  
Generally I want to give them the qualifying information and have them
go away.

> If it were possible to hook the message deletion functions in cyrus 
> to move things to a different place in the hierarchy and then control
> expiry on those differently than the regular folders, it would 
> probably be sufficient.  But that requires code and I don't have the 
> skills to write it.

What you are talking about is "tiered storage".  That has been talked
about in the past - I don't know if anyone has implemented it.

> Certainly not super featureful but frankly
> when the lawyers want something, I just dump mail files on them and 
> let them sort it out.

Exactly!  So perhaps just dump them out of the system in the first
place.


-- 
Adam Tauno Williams  GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: how to deal with mail retention/archival.

[Adam Tauno Williams via Info-cyrus]

On Fri, 2016-08-26 at 16:13 +, Shawn Bakhtiar via Info-cyrus wrote:
> > On Aug 26, 2016, at 8:35 AM, Giuseppe Ravasio (LU) via Info-cyrus <
> > info-cyrus@lists.andrew.cmu.edu> wrote:
> > I saw that someone proposed to make a sort of abuse of delayed
> > expunge,
> > but I think that in order to comply with regulatory retention 
> > should be better considering some specific software.
> I don't see how using delayed expunge would really be consider abuse,
> the documentation makes mention of its use for this very reason.

+1


> We use rsync to make a duplicate of the email spool to a file server
> at regular intervals, which eventually makes its way to tape.


Same here.  And always_bcc to a shared folder which is dumped to an
MBOX file via fetchmail at an interval.  Those can be archived or even
shipped off-site.

> Although we don't have regulatory requirements I've had to do a few
> recoveries and have done so without problem. 

I always advise people to be hesitant about "we don't have regulatory
requirements" as if you are a legal corporation of any kind, in almost
all of the 50 states [United States], you are under data retention
rules - even if you don't know it.  Which you will discover when you
are involved in a law suit - saying "uhh... yeah, we don't have those e
-mails" will not be good.

> > Finding something in the delayed_expunge folders after many years
> > of archive will absolutely be a nightmare!

Most states [again the United States] allow a corporation to have on
file a documented data retention policy that states how long you retain
e-mails;  which if you comply with you will be OK.  The policy just
needs to be 'reasonable'.  For example: where I work we say 120 days. 
 No need - at least for legal reasons - to have years of archives.

Obviously requirements vary by industry - but almost everyone is
actually under some kind of requirement.

-- 
Adam Tauno Williams  GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: how to deal with mail retention/archival.

[Adam Tauno Williams via Info-cyrus]

On Fri, 2016-08-26 at 10:07 -0400, Alvin Starr via Info-cyrus wrote:
> Well the MTA still does not deal with archival because it will need
> to be passed through to Yet Another MDA to handle the archival and
> management process.

I'm not sure what you mean.  You can archive to a 'shared' folder or
into an MBOX to be processed by something which rotates content.

> For the pure archival of the input/output stream including duplicate
> deliveries and all spam always_bcc into YAMDA would work.

always_bcc and delayed expunge work for us.

> In my thinking Cyrus is responsible for the storage and management of
> email so archival would be a part of that process.

It has to be the MTAs responsibility as Cyrus very possibly does not
see *sent* mail; or messages which are somehow otherwise routed.

-- 
Adam Tauno Williams  GPG D95ED383
OpenGroupware Developer 



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: [POLL] Cyrus ACLs and group names

[Adam Tauno Williams via Info-cyrus]

On Tue, 2015-11-17 at 07:40 +1100, Bron Gondwana via Info-cyrus wrote:
> For those of you using Cyrus with group ACLs, how are your groups
> named?
> I know with the auth_unix backend, they are
> 'group:'.  What I've seen from CMU's groups is that they
> are of the form ':'.

Ours are group:

-- 
Adam Tauno Williams  GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: delprune on a single mailbox

[Adam Tauno Williams via Info-cyrus]

globally in cyrus.conf delprune is set to
> > > > delprunecmd="/usr/sbin/cyrus expire -E 1 -X 7 -D 7"
> > > > at=0501
> > > > For a single mailbox I don't want to keep deleted mails for 7
> > > > days,
> > > > but
> > > > expire them immediately or once a day per cron. How to do that?
> > > Forogt to say that delete_mode and expunge_mode is set to
> > > delayed.
> > > Via cron this should work for an immediate cleanup/expire:
> > You can set an expire annotation per mailbox.  
> How do I do that? From cyr_expire manpage:
> "The value of the /vendor/cmu/cyrus-imapd/expire annotation is
> inherited by all children of the given mailbox, so an entire mailbox
> tree can be expired by seting a single annotation on the root of that
> tree. If a mailbox does not have a /vendor/cmu/cyrus-imapd/expire
> annotation set on it (or does not inherit one), then no messages are
> expired from the mailbox."

Via cyradm -

cyrus.example.com> mboxcfg user.adam expire 365 
cyrus.example.com> info user.adam 
{user.adam}: 
  condstore: false 
  duplicatedeliver: false 
  expire: 365 
  lastpop:  
  lastupdate: 13-Aug-2008 19:37:31 -0400 
  partition: default 
  sharedseen: false 
  size: 12325671

AFAIK the annotations supported by cyradm/mboxcfg are:

* comment – A free-form text comment or description to be attached to
the mailbox.
* condstore – This annotation is only supported in the 2.3.x release
series starting with 2.3.3 although its use is not recommended until
2.3.8. As of the 2.4.x release series CONDSTORE functionality is
enabled on all mailboxes regardless of annotation and attempting to set
this annotation will result in a permission denied message. On releases
where this annotation is supported setting a value of “true” will
enable CONDSTORE functionality1.
* expire – If an expire value is provided messages will be
automatically deleted from the mailbox once the specified number of
days has elapsed.
* news2mail - 
* sharedseen - Enables the use of a shared \Seen flag on messages
rather than a per-user \Seen flag. The 's' right in the mailbox ACL
still controls whether a user can set the shared \Seen flag.
* sieve – In the case of a shared folder the “sieve” parameter
specifies the name of a global SIEVE script that will be used for every
message delivered to the folder.  This value is ignored for personal
mailboxes (mailboxes including and subordinate to a user's INBOX).
* squat – Flags the mailbox to be included for indexing when the SQUAT
process performs index generation.


> But is it possible to expunge a message immediately when it's deleted
> by client and not with the next expire run?

Not if delayed expunge is enabled AFAIK; that would defeat the purpose.

-- 
Adam Tauno Williams  GPG D95ED383
OpenGroupware Developer 



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: delprune on a single mailbox

[Adam Tauno Williams via Info-cyrus]

On Sun, 2015-11-01 at 14:40 +0100, Marcus Schopen via Info-cyrus wrote:
> Am Sonntag, den 01.11.2015, 13:35 +0100 schrieb Marcus Schopen via
> Info-cyrus:
> > Hi,
> > globally in cyrus.conf delprune is set to
> > delprunecmd="/usr/sbin/cyrus expire -E 1 -X 7 -D 7" at=0501
> > For a single mailbox I don't want to keep deleted mails for 7 days,
> > but
> > expire them immediately or once a day per cron. How to do that?
> Forogt to say that delete_mode and expunge_mode is set to delayed.
> Via cron this should work for an immediate cleanup/expire:

You can set an expire annotation per mailbox.  Downside is that I
believe the annotation will be 'inherited' but subordinate mailboxes;
which stinks for some use-cases.


> su - cyrus -c "/usr/sbin/cyrus expire -E 1 -X 0 -D 0 -v -p
> user.mailboxname"

FYI, I believe with the very latest Cyrus the "su -" is unnecessary as
it will automatically handle the context change when run as root.


-- 
Adam Tauno Williams  GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus