date:20160929

Re: command line deletion of files

2016-09-29 Thread Bron Gondwana via Info-cyrus

Ahh, and reading on I see that this exists already :)

On Fri, 30 Sep 2016, at 08:02, Bron Gondwana wrote:
> You're the reason we can't have nice things :( rm + reconstruct will bite you 
> one upgrade for sure.
> 
> A dry run option to ipurge sounds like a great idea. We just always use IMAP 
> to do admin on Cyrus, but I can see a case for improving ipurge.
> 
> On Fri, 30 Sep 2016, at 01:04, Vladislav Kurz via Info-cyrus wrote:
> > On 09/29/16 16:32, Patrick Boutilier via Info-cyrus wrote:
> > > On 09/29/2016 11:27 AM, Shawn Bakhtiar via Info-cyrus wrote:
> > >> Good morning,
> > >>
> > >> trying to get rid of some emails that have large attachments (i.e.
> > >> videos sent over email, or cd images, etc...)
> > >>
> > >> Would it be proper to
> > >>
> > >> rm -rf /var/spool/imap/u/username/mailbox/4321.
> > >>
> > >> then
> > >>
> > >> reconstruct -rf user.username
> > >>
> > >> Or is there a more "proper" way using cyrus?
> > > 
> > > Not sure about deleting a single message but you can use ipurge to
> > > delete messages based on size. Good to use in a script to parses the
> > > mail spool.
> > > 
> > 
> > rm + reconstruct is IMHO ok, we use that for trash/spam cleanup,
> > antivirus checks, and similar things.
> > 
> > ipurge is nice but I would really appreciate if it had a --dry-run
> > option. I'm never sure what it will really delete.
> > 
> > 
> > -- 
> > Best Regards
> > Vladislav Kurz
> > 
> > 
> > Cyrus Home Page: http://www.cyrusimap.org/
> > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> > To Unsubscribe:
> > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> 
> -- 
>   Bron Gondwana
>   br...@fastmail.fm


-- 
  Bron Gondwana
  br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: command line deletion of files

2016-09-29 Thread Bron Gondwana via Info-cyrus

You're the reason we can't have nice things :( rm + reconstruct will bite you 
one upgrade for sure.

A dry run option to ipurge sounds like a great idea. We just always use IMAP to 
do admin on Cyrus, but I can see a case for improving ipurge.

On Fri, 30 Sep 2016, at 01:04, Vladislav Kurz via Info-cyrus wrote:
> On 09/29/16 16:32, Patrick Boutilier via Info-cyrus wrote:
> > On 09/29/2016 11:27 AM, Shawn Bakhtiar via Info-cyrus wrote:
> >> Good morning,
> >>
> >> trying to get rid of some emails that have large attachments (i.e.
> >> videos sent over email, or cd images, etc...)
> >>
> >> Would it be proper to
> >>
> >> rm -rf /var/spool/imap/u/username/mailbox/4321.
> >>
> >> then
> >>
> >> reconstruct -rf user.username
> >>
> >> Or is there a more "proper" way using cyrus?
> > 
> > Not sure about deleting a single message but you can use ipurge to
> > delete messages based on size. Good to use in a script to parses the
> > mail spool.
> > 
> 
> rm + reconstruct is IMHO ok, we use that for trash/spam cleanup,
> antivirus checks, and similar things.
> 
> ipurge is nice but I would really appreciate if it had a --dry-run
> option. I'm never sure what it will really delete.
> 
> 
> -- 
> Best Regards
> Vladislav Kurz
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


-- 
  Bron Gondwana
  br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Twoskip DB files broken

2016-09-29 Thread Bron Gondwana via Info-cyrus

A couple of things.

1. Why are you doing this? What do you hope to achieve?

2. Possibly kolab's Cyrus configuration stores files in other paths (tmpfs, 
data dirs) which are Berkeley dbs and don't expect their environment to be 
trashed under them.

On Thu, 29 Sep 2016, at 23:47, Tobias Brunner via Info-cyrus wrote:
> Hi,
> 
> I've discovered an odd behaviour which I don't understand: After a
> completely fresh Kolab 16 install on CentOS 7
> (cyrus-imapd-2.5.9.27-5.1.el7.kolab_wf.x86_64) everything looks fine. I
> can create mailboxes, stop/start Cyrus, all works as it should do. The
> contents of /var/lib/imap looks like this:
> 
> -rw--- 1 cyrus mail 3.8K Sep 29 14:48 annotations.db
> drwxr-x--- 2 cyrus mail6 Sep 22 17:25 backup
> drwxr-x--- 2 cyrus mail   22 Sep 29 14:48 db
> drwx-- 2 cyrus mail   46 Sep 29 14:48 db.backup1
> -rw--- 1 cyrus mail  336 Sep 29 14:48 deliver.db
> drwx-- 3 cyrus mail   14 Sep 29 14:48 domain
> drwx-- 5 cyrus mail   35 Sep 29 14:48 lock
> drwxr-x--- 2 cyrus mail6 Sep 22 17:25 log
> -rw--- 1 cyrus mail 3.7K Sep 29 14:48 mailboxes.db
> drwxr-x--- 2 cyrus mail6 Sep 22 17:25 md5
> drwxr-x--- 2 cyrus mail6 Sep 22 17:25 meta
> drwxr-x--- 2 cyrus mail6 Sep 22 17:25 msg
> drwxr-x--- 2 cyrus mail   18 Sep 29 14:56 proc
> drwxr-x--- 2 cyrus mail   37 Sep 29 14:48 ptclient
> drwxr-x--- 2 cyrus mail6 Sep 22 17:25 quota
> drwxr-x--- 2 cyrus mail   51 Sep 29 14:48 rpm
> drwxr-x--- 2 cyrus mail6 Sep 22 17:25 sieve
> drwxr-x--- 2 cyrus mail 4.0K Sep 29 14:56 socket
> -rw--- 1 cyrus mail  336 Sep 29 14:48 statuscache.db
> drwxr-x--- 2 cyrus mail6 Sep 22 17:25 sync
> -rw--- 1 cyrus mail  768 Sep 29 14:48 tls_sessions.db
> drwxr-x--- 2 cyrus mail6 Sep 22 17:25 user
> 
> Then I do:
> 
> 1. Stop Cyrus: systemctl stop cyrus-imapd. Log says:
> 
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: attempting clean
> shutdown on signal
> Sep 29 14:57:59 kolabv.vagrant.dev pop3[16965]: graceful shutdown
> initiated by unexpected process 1 (/usr/lib/systemd/systemd
> --switched-root --system --deserialize 21)
> Sep 29 14:57:59 kolabv.vagrant.dev pop3[16971]: graceful shutdown
> initiated by unexpected process 1 (/usr/lib/systemd/systemd
> --switched-root --system --deserialize 21)
> Sep 29 14:57:59 kolabv.vagrant.dev lmtpunix[16967]: graceful shutdown
> initiated by unexpected process 1 (/usr/lib/systemd/systemd
> --switched-root --system --deserialize 21)
> Sep 29 14:57:59 kolabv.vagrant.dev systemd[1]: Stopping Cyrus-imapd
> IMAP/POP3 email server...
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:imap path:/usr/lib/cyrus-imapd/imapd age:561.385s pid:16959 exited,
> status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:imap path:/usr/lib/cyrus-imapd/imapd age:561.385s pid:16961 exited,
> status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.385s pid:16963 exited,
> status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.385s pid:16964 exited,
> status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:pop3s path:/usr/lib/cyrus-imapd/pop3d age:561.385s pid:16966
> exited, status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:imap path:/usr/lib/cyrus-imapd/imapd age:561.387s pid:16958 exited,
> status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:imap path:/usr/lib/cyrus-imapd/imapd age:561.389s pid:16960 exited,
> status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.389s pid:16965 exited,
> status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:lmtpunix path:/usr/lib/cyrus-imapd/lmtpd age:561.388s pid:16967
> exited, status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:notify path:/usr/lib/cyrus-imapd/notifyd age:561.388s pid:16968
> exited, status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.388s pid:16969 exited,
> status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.388s pid:16970
> signaled to death by signal 15 (Terminated)
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.388s pid:16971 exited,
> status 75
> Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
> name:pop3s path:/usr/lib/cyrus-imapd/pop3d age:561.388s pid:16972
> exited, status 75
> Sep 29 14:57:59 kolabv.vagrant.dev systemd[1]: Stopped Cyrus-imapd
> IMAP/POP3 email server.
> 
> 2. Remove all content of /var/lib/imap: rm -rf /var/l

Re: command line deletion of files

2016-09-29 Thread John Wade via Info-cyrus

We use a perl scrip that does similar process to delete mass phishing or 
malware email messages from the mail spool. The script locates the 
sequence number for the message based on the message ID/filename, and 
then issues the IMAP delete and expunge commands.


Relevant section of code below  (This is called from a subroutine that 
uses File::Find to locate the files in the file system, but it could be 
called with a simple list of files.) Advantage over ipurge is that this 
is not based on date or size but a specific set of files.


Enjoy,
John

use Net::IMAP::Simple;
use Cyrus::IMAP::Admin;

sub imapdeletemessage {
( my $path ) = @_;
## Extract out foldername and filename (UID) from path
$path =~ /\/var\/spool\/imap\/\d\/(.*)\/(\d+.)/;
my $foldername = $1;
my $messageuid = $2;
return "Error: Unable to identify folder or message for $path" if ( 
! defined($foldername) || ! defined ($messageuid) );

$foldername =~ s/\//\./g;
$messageuid =~ s/\.$//g;
## Give ourselves permissions
if ( my $err = &setimapacl($foldername, 'rte')) {
   return "$err";
}

# Create the IMAP object
my $imap = Net::IMAP::Simple->new('localhost') ||
   die "Unable to connect to IMAP: $Net::IMAP::Simple::errstr\n";

# Log on
if(!$imap->login($CYRUSADMIN,$CYRUSADMINPW)){
print STDERR "Login failed: " . $imap->errstr . "\n";
exit(64);
}

## Select folder and get number of messages
my $nm = $imap->select($foldername);
if ( $imap->waserr || ! defined($nm)  ) {
   &setimapacl($foldername, 'none');
   return "ERROR: unable to select $foldername: " . $imap->errstr;
}

### Now get Sequence Number for Message ID
my @seq = $imap->seq($messageuid);
if ( ! defined($seq[0])) {
   &setimapacl($foldername, 'none');
   return "ERROR: No sequence number for UID $messageuid 
$foldername, probably deleted\n";

}
# print "seq for uid $messageuid is $seq[0]\n";

## Delete the message
if ( ! $imap->delete( $seq[0] )) {
   &setimapacl($foldername, 'none');
   return "ERROR: Unable to delete $messageuid from $foldername: " 
. $imap->errstr;

}

## Expunge the mailbox to be safe
my $expunged = $imap->expunge_mailbox( $foldername );
if  ( $imap->waserr ) {
&setimapacl($foldername, 'none');
return "ERROR: unabled to expunge $foldername: " . $imap->errstr;
}
$imap->quit;
print "IMAP Deleted: UID:$messageuid from folder:$foldername\n";
print LOG "IMAP Deleted: UID:$messageuid from folder:$foldername\n" 
if ( $logfile);


## Take away our permissions
if ( my $err = &setimapacl($foldername, 'none')) {
   return "$err";
}
return 0;
};


On 09/29/2016 11:27 AM, Shawn Bakhtiar via Info-cyrus wrote:

Good morning,

trying to get rid of some emails that have large attachments (i.e. 
videos sent over email, or cd images, etc...)


Would it be proper to

rm -rf /var/spool/imap/u/username/mailbox/4321.

then

reconstruct -rf user.username

Or is there a more "proper" way using cyrus?





Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: watching and processing a Spam folder for each user

2016-09-29 Thread Alvin Starr via Info-cyrus


Take a look at sa-learn-cyrus.

Its a perl program that will traverse users spam/ham boxes and process them.

It may not be exactly what you want but would be a good jumping off point.


On 09/29/2016 02:48 PM, Adam Tauno Williams via Info-cyrus wrote:

While I can see this being a neat built-in feature of a mail server
like Cyrus IMAP, I doubt it exists.  I'd be happy to be corrected.

Good old fecthmail.

fetchmail --verbose --all --norewrite  \
   --folder 'user.awilliam.SPAM' --mda '/usr/bin/sa-learn --spam'


I wonder if such a beast exists.  I'd love any pointers if anyone
knows of such.

Yes, you probably already have it installed.





Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


--
Alvin Starr   ||   voice: (905)513-7688
Netvel Inc.   ||   Cell:  (416)806-0133
al...@netvel.net  ||


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: watching and processing a Spam folder for each user

2016-09-29 Thread Adam Tauno Williams via Info-cyrus

> While I can see this being a neat built-in feature of a mail server
> like Cyrus IMAP, I doubt it exists.  I'd be happy to be corrected.

Good old fecthmail.

fetchmail --verbose --all --norewrite  \
  --folder 'user.awilliam.SPAM' --mda '/usr/bin/sa-learn --spam'

> I wonder if such a beast exists.  I'd love any pointers if anyone
> knows of such.

Yes, you probably already have it installed.


-- 
Adam Tauno Williams  GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA



signature.asc
Description: This is a digitally signed message part

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: watching and processing a Spam folder for each user

2016-09-29 Thread Jason L Tibbitts III via Info-cyrus

> "PB" == Patrick Boutilier via Info-cyrus 
>  writes:

PB> Only problem with that is users always seem to report some stuff as
PB> spam when it clearly isn't. :-)

True, but at least it's only a statistical thing.  You could easily
extract the From: headers and blacklist them, but that would be far more
problematic, I think.

What I really want to be able to do is to periodically plow through all
of the unread messages in everyone's mailboxes and pass them back
through spamassassin.  The blacklists lag the spam runs and it would be
nice to at least get rid of stuff which wasn't blocked initially but
which would be blocked later.  I guess I need to write some more code.

 - J<

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: watching and processing a Spam folder for each user

2016-09-29 Thread Brian J. Murrell via Info-cyrus

On Thu, 2016-09-29 at 12:25 -0300, Patrick Boutilier via Info-cyrus
wrote:
> 
> Only problem with that is users always seem to report some stuff as
> spam 
> when it clearly isn't. :-)

That's fine.  They are only poisoning their own well if they do since
each user has their own Bayes database.

But if you really did want to babysit your users, and perhaps enroll
them in some remedial e-mail/spam handling training, you could also
have whatever this "tool" that I am looking for copy anything marked as
spam (under a certain threshold even) to an admin for investigation.

You see, this "tool" that I am looking for (and can write myself if I
need to -- I'm just looking to not re-invent wheels if I don't have to)
is not so much the "what to do" with spam part as much as it's the
monitoring of everyone's [Not][Spam] mailboxes and forward whatever is
in them to the "what to do" part.

Cheers,
b.

signature.asc
Description: This is a digitally signed message part

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: watching and processing a Spam folder for each user

2016-09-29 Thread Patrick Boutilier via Info-cyrus


On 09/29/2016 12:12 PM, Jason L Tibbitts III via Info-cyrus wrote:

"BJM" == Brian J Murrell via Info-cyrus  
writes:


BJM> So leaving out the latter part (the per-user database and handling,
BJM> etc.) I wonder what, if anything exists to monitor the Spam (and
BJM> NotSpam) folders for all users.

I have a system which sucks things out of everyone's "confirmed-spam"
folders and feeds it to spamassassin on each filtering host.  It's in
Perl (back from when I remembered how to do perl) and is probably
unpleasant.

https://www.math.uh.edu/~tibbs/spamsuck/

Run spamsuck to pull down all of the spam folders into mbox files and
empty them.  Note that the "spamkill" user (or whatever you choose to
call it) needs ACLs ("lrte", I think) on the confirmed-spam folder, so
your user creation process or sommething run as your admin user needs to
set that up.  There's no reason you couldn't pull out ham folders as
well.

Run spamlearn on each filtering host to feed the sucked spam to
sa_learn.  This updates the global bayes databases, not anything
personal to each user.

I don't bother to do this all automatically, but you could.



Only problem with that is users always seem to report some stuff as spam 
when it clearly isn't. :-)






 - J<

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



<>
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: command line deletion of files

2016-09-29 Thread Vladislav Kurz via Info-cyrus

On 09/29/16 17:14, Wolfgang Breyha via Info-cyrus wrote:
> Vladislav Kurz via Info-cyrus wrote on 29/09/16 17:04:
>> ipurge is nice but I would really appreciate if it had a --dry-run
>> option. I'm never sure what it will really delete.
> 
> It got one in 2.5.9.
> 
> Greetings, Wolfgang
> 
Ah, thank you very much. +1 (like, thumbsup, etc ;)

-- 
Best regards
Vladislav Kurz


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: command line deletion of files

2016-09-29 Thread Wolfgang Breyha via Info-cyrus

Vladislav Kurz via Info-cyrus wrote on 29/09/16 17:04:
> ipurge is nice but I would really appreciate if it had a --dry-run
> option. I'm never sure what it will really delete.

It got one in 2.5.9.

Greetings, Wolfgang
-- 
Wolfgang Breyha  | http://www.blafasel.at/
Vienna University Computer Center | Austria


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: watching and processing a Spam folder for each user

2016-09-29 Thread Jason L Tibbitts III via Info-cyrus

> "BJM" == Brian J Murrell via Info-cyrus  
> writes:

BJM> So leaving out the latter part (the per-user database and handling,
BJM> etc.) I wonder what, if anything exists to monitor the Spam (and
BJM> NotSpam) folders for all users.

I have a system which sucks things out of everyone's "confirmed-spam"
folders and feeds it to spamassassin on each filtering host.  It's in
Perl (back from when I remembered how to do perl) and is probably
unpleasant.

https://www.math.uh.edu/~tibbs/spamsuck/

Run spamsuck to pull down all of the spam folders into mbox files and
empty them.  Note that the "spamkill" user (or whatever you choose to
call it) needs ACLs ("lrte", I think) on the confirmed-spam folder, so
your user creation process or sommething run as your admin user needs to
set that up.  There's no reason you couldn't pull out ham folders as
well.

Run spamlearn on each filtering host to feed the sucked spam to
sa_learn.  This updates the global bayes databases, not anything
personal to each user.

I don't bother to do this all automatically, but you could.

 - J<

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: command line deletion of files

2016-09-29 Thread Vladislav Kurz via Info-cyrus

On 09/29/16 16:32, Patrick Boutilier via Info-cyrus wrote:
> On 09/29/2016 11:27 AM, Shawn Bakhtiar via Info-cyrus wrote:
>> Good morning,
>>
>> trying to get rid of some emails that have large attachments (i.e.
>> videos sent over email, or cd images, etc...)
>>
>> Would it be proper to
>>
>> rm -rf /var/spool/imap/u/username/mailbox/4321.
>>
>> then
>>
>> reconstruct -rf user.username
>>
>> Or is there a more "proper" way using cyrus?
> 
> Not sure about deleting a single message but you can use ipurge to
> delete messages based on size. Good to use in a script to parses the
> mail spool.
> 

rm + reconstruct is IMHO ok, we use that for trash/spam cleanup,
antivirus checks, and similar things.

ipurge is nice but I would really appreciate if it had a --dry-run
option. I'm never sure what it will really delete.


-- 
Best Regards
Vladislav Kurz


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

watching and processing a Spam folder for each user

2016-09-29 Thread Brian J. Murrell via Info-cyrus

I have experienced e-mail systems where each user has a "Spam" (and
"NotSpam" on some) folder in their folder hierarchy to which they can
simply move spam to have it classified as spam for them personally (per
user Bayes databases for example).

So leaving out the latter part (the per-user database and handling,
etc.) I wonder what, if anything exists to monitor the Spam (and
NotSpam) folders for all users.

While I can see this being a neat built-in feature of a mail server
like Cyrus IMAP, I doubt it exists.  I'd be happy to be corrected.

But short of that, I can imagine a monitoring process which, using
IMAP, opens a handle to the Spam (and NotSpam) for every user and takes
action to classify anything that shows up in those folders accordingly,
by perhaps handing the items that do show up to another
"classification" process (i.e. spamassassin perhaps).

I wonder if such a beast exists.  I'd love any pointers if anyone knows
of such.

Cheers,
b.


signature.asc
Description: This is a digitally signed message part

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: command line deletion of files

2016-09-29 Thread Dan White via Info-cyrus


On 09/29/16 14:27 +, Shawn Bakhtiar via Info-cyrus wrote:

trying to get rid of some emails that have large attachments (i.e. videos sent 
over email, or cd images, etc...)

Would it be proper to

rm -rf /var/spool/imap/u/username/mailbox/4321.

then

reconstruct -rf user.username

Or is there a more "proper" way using cyrus?


I've found mutt to be useful for this type of maintenance, which can sort
messages by size, and can delete ranges. If you don't have access to user
passwords, set up a 'proxyservers' authz identity to access their
mailboxes.

--
Dan White

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: command line deletion of files

2016-09-29 Thread Patrick Boutilier via Info-cyrus


On 09/29/2016 11:27 AM, Shawn Bakhtiar via Info-cyrus wrote:

Good morning,

trying to get rid of some emails that have large attachments (i.e. videos sent 
over email, or cd images, etc...)

Would it be proper to

rm -rf /var/spool/imap/u/username/mailbox/4321.

then

reconstruct -rf user.username

Or is there a more "proper" way using cyrus?


Not sure about deleting a single message but you can use ipurge to 
delete messages based on size. Good to use in a script to parses the 
mail spool.






IPURGE(8) 
IPURGE(8)


 *

NAME
   ipurge - delete mail from IMAP mailbox or partition based on age 
or size


SYNOPSIS
   ipurge [ -f ] [ -C config-file ] [ -x ] [ -X ] [ -i ] [ -s ] [ -o ]
  [ -d days | -b bytes | -k Kbytes | -m Mbytes ]
  [ mailbox-pattern...  ]

DESCRIPTION
   Ipurge  deletes messages from the mailbox(es) specified by 
mailbox-pattern that are older or larger than specified by the -d, -b, 
-k or -m options.  If no mailbox-pattern
   is given, ipurge works on all mailboxes.  If the -x option is 
given, the message age and size MUST match exactly those specified by 
-d, -b, -k or -m.  The are no  default

   values, and at least one of -d, -b, -k or -m MUST be specified.

   Ipurge  by default only deletes mail below shared folders, which 
means that mails in mailbox(es) below INBOX.* and user.* stay untouched. 
Use the option -f to also delete

   mail in mailbox(es) below these folders.

   Ipurge reads its configuration options out of the imapd.conf(5) 
file unless specified otherwise by -C.


OPTIONS
   -f Force deletion of mail in all mailboxes.

   -C config-file
  Read configuration options from config-file.

   -d days
  Age of message in days.

   -b bytes
  Size of message in bytes.

   -k Kbytes
  Size of message in Kbytes (2^10 bytes).

   -m Mbytes
  Size of message in Mbytes (2^20 bytes).

   -x Perform an exact match on age or size (instead of older 
or larger).


   -X Use delivery time instead of Date: header for date matches

   -i Invert match logic: -x means not equal, date is for 
newer, size is for smaller


   -s Skip over messages that have the \Flagged flag set.

   -o Only purge messages that have the \Deleted flag set.








Thanks,
Shawn


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



<>
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

command line deletion of files

2016-09-29 Thread Shawn Bakhtiar via Info-cyrus

Good morning,

trying to get rid of some emails that have large attachments (i.e. videos sent 
over email, or cd images, etc...)

Would it be proper to 

rm -rf /var/spool/imap/u/username/mailbox/4321.

then

reconstruct -rf user.username

Or is there a more "proper" way using cyrus?

Thanks,
Shawn


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Twoskip DB files broken

2016-09-29 Thread Tobias Brunner via Info-cyrus

Hi,

I've discovered an odd behaviour which I don't understand: After a
completely fresh Kolab 16 install on CentOS 7
(cyrus-imapd-2.5.9.27-5.1.el7.kolab_wf.x86_64) everything looks fine. I
can create mailboxes, stop/start Cyrus, all works as it should do. The
contents of /var/lib/imap looks like this:

-rw--- 1 cyrus mail 3.8K Sep 29 14:48 annotations.db
drwxr-x--- 2 cyrus mail6 Sep 22 17:25 backup
drwxr-x--- 2 cyrus mail   22 Sep 29 14:48 db
drwx-- 2 cyrus mail   46 Sep 29 14:48 db.backup1
-rw--- 1 cyrus mail  336 Sep 29 14:48 deliver.db
drwx-- 3 cyrus mail   14 Sep 29 14:48 domain
drwx-- 5 cyrus mail   35 Sep 29 14:48 lock
drwxr-x--- 2 cyrus mail6 Sep 22 17:25 log
-rw--- 1 cyrus mail 3.7K Sep 29 14:48 mailboxes.db
drwxr-x--- 2 cyrus mail6 Sep 22 17:25 md5
drwxr-x--- 2 cyrus mail6 Sep 22 17:25 meta
drwxr-x--- 2 cyrus mail6 Sep 22 17:25 msg
drwxr-x--- 2 cyrus mail   18 Sep 29 14:56 proc
drwxr-x--- 2 cyrus mail   37 Sep 29 14:48 ptclient
drwxr-x--- 2 cyrus mail6 Sep 22 17:25 quota
drwxr-x--- 2 cyrus mail   51 Sep 29 14:48 rpm
drwxr-x--- 2 cyrus mail6 Sep 22 17:25 sieve
drwxr-x--- 2 cyrus mail 4.0K Sep 29 14:56 socket
-rw--- 1 cyrus mail  336 Sep 29 14:48 statuscache.db
drwxr-x--- 2 cyrus mail6 Sep 22 17:25 sync
-rw--- 1 cyrus mail  768 Sep 29 14:48 tls_sessions.db
drwxr-x--- 2 cyrus mail6 Sep 22 17:25 user

Then I do:

1. Stop Cyrus: systemctl stop cyrus-imapd. Log says:

Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: attempting clean
shutdown on signal
Sep 29 14:57:59 kolabv.vagrant.dev pop3[16965]: graceful shutdown
initiated by unexpected process 1 (/usr/lib/systemd/systemd
--switched-root --system --deserialize 21)
Sep 29 14:57:59 kolabv.vagrant.dev pop3[16971]: graceful shutdown
initiated by unexpected process 1 (/usr/lib/systemd/systemd
--switched-root --system --deserialize 21)
Sep 29 14:57:59 kolabv.vagrant.dev lmtpunix[16967]: graceful shutdown
initiated by unexpected process 1 (/usr/lib/systemd/systemd
--switched-root --system --deserialize 21)
Sep 29 14:57:59 kolabv.vagrant.dev systemd[1]: Stopping Cyrus-imapd
IMAP/POP3 email server...
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:imap path:/usr/lib/cyrus-imapd/imapd age:561.385s pid:16959 exited,
status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:imap path:/usr/lib/cyrus-imapd/imapd age:561.385s pid:16961 exited,
status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.385s pid:16963 exited,
status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.385s pid:16964 exited,
status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:pop3s path:/usr/lib/cyrus-imapd/pop3d age:561.385s pid:16966
exited, status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:imap path:/usr/lib/cyrus-imapd/imapd age:561.387s pid:16958 exited,
status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:imap path:/usr/lib/cyrus-imapd/imapd age:561.389s pid:16960 exited,
status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.389s pid:16965 exited,
status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:lmtpunix path:/usr/lib/cyrus-imapd/lmtpd age:561.388s pid:16967
exited, status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:notify path:/usr/lib/cyrus-imapd/notifyd age:561.388s pid:16968
exited, status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.388s pid:16969 exited,
status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.388s pid:16970
signaled to death by signal 15 (Terminated)
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:pop3 path:/usr/lib/cyrus-imapd/pop3d age:561.388s pid:16971 exited,
status 75
Sep 29 14:57:59 kolabv.vagrant.dev master[16944]: process type:SERVICE
name:pop3s path:/usr/lib/cyrus-imapd/pop3d age:561.388s pid:16972
exited, status 75
Sep 29 14:57:59 kolabv.vagrant.dev systemd[1]: Stopped Cyrus-imapd
IMAP/POP3 email server.

2. Remove all content of /var/lib/imap: rm -rf /var/lib/imap/*
3. Start Cyrus, which automatically recreates the folder structure under
/var/lib/imap. The folder content:

-rw--- 1 cyrus mail 3.8K Sep 29 15:00 annotations.db
drwxr-xr-x 2 cyrus mail   22 Sep 29 15:00 db
drwx-- 2 cyrus mail   46 Sep 29 15:00 db.backup1
-rw--- 1 cyrus mail  336 Sep 29 15:00 deliver.db
drwx-- 3 cyrus mail   14 Sep 29 15:00 domain
drwx-- 5 cyrus mail   35 Sep 29 15:00 lock
drwxr-xr-x 2 cyrus mail6 Sep 29 15:00 log
-rw--- 1 cyrus mail 3.6K Sep 29 15:

Huge performance problems after updating from 2.4 to 2.5.9

2016-09-29 Thread Wolfgang Breyha via Info-cyrus

Hi!

A can add another story of that type, but with different setup:

We already migrated to 2.5.7 on our ten backends some month ago step by step
and upgraded to 2.5.9 lately. We never had any performance issues on them. All
of them have done a full "reconstruct -V max" and special-use metadata is set
to step in for static xlist missing in 2.5 frontends.

Yesterday I did the final step by updating our mupdate server and the murder
frontends. No problems on the mupdate server so far.

The configs are mostly the same like for 2.4 (except for the changed names).
The only key change done was from skiplist to twoskip for all databases:
annotation_db: twoskip
duplicate_db: twoskip
ptscache_db: twoskip
mboxlist_db: twoskip
seenstate_db: twoskip
statuscache_db: twoskip
subscription_db: flat
tls_sessions_db: twoskip
userdeny_db: flat

All of them are placed in /var/spool/imap/config except statuscache and
tlscache which reside in /dev/shm.

After migrating the frontends yesterday evening I already recognized a higher
load and decided to keep an eye on it.

Our 3 frontends are:
CentOS 6 (vmware based)
16GB RAM
4 CPUs

They usually have ~4000-4500 concurrent imap(s) connections at peak time and
had a load ~1-3 with cyrus 2.4. swap was not used.

Today in the morning we had the same connection count with load ~100-250 with
cyrus 2.5.9 and twoskip. no swap used. top shows a huge amount of imap
processes in running state. Another thing I recognized was that changes in
mailboxes.db on backends reached the mupdate server, but didn't come through
to the frontends anymore even after 15 minutes.

I changed mailboxes.db and tlscache back to skiplist and load went down to
5-15. So, much better but still 5 times as high as with 2.4 and still causing
headaches.

Currently I have no idea what causes the load besides twoskip (which I can
live without since skiplist never caused us any troubles).

I compared "USAGE" loglines from two days (2.4) ago to 2.5 with skiplist and
twoskip.

2.4   : user: 0.384322, sys: 0.029056 (23538 USAGE lines, 09:00-10:00)
2.4   : user: 0.353515, sys: 0.030814 (25790 USAGE lines, 13:00-14:00)
2.5 ts: user: 0.903491, sys: 0.028196 (23429 ..., 08:00-09:00)
2.5 ts: user: 1.130391, sys: 0.032894 (27709 ..., 09:00-10:00)
2.5 sl: user: 0.864270, sys: 0.026580 (29567 ..., 13:00-14:00)
2.5 sl: user: 1.077462, sys: 0.032952 (25879 ..., 14:00-15:00)

Comparing all three I wonder why skiplist makes such a big difference.

lmtpd shows now relevant difference for all three.

Most likely I will go for horizontal scaling putting a 4th frontend in the line.

Greetings, Wolfgang
-- 
Wolfgang Breyha  | http://www.blafasel.at/
Vienna University Computer Center | Austria


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: why sieve's "reject" is incompatible with imap4flags actions ?

2016-09-29 Thread Ken Murchison via Info-cyrus




On 09/28/2016 10:52 AM, Deniss via Info-cyrus wrote:

hello,

in sieve/message.c in do_reject() all imap4flags actions are
incompatible with reject action.

Why ? imap4flags does no delivery indeed.

what is a reason to ban "redirect" action with "reject" in rfc5429

This orginally comes from a SHOULD in RFC 3028, Section 2.10.4:
"Implementations SHOULD prohibit reject when used with other actions."

This was restriction was relaxed somewhat in RFC 5429, Section 2.4:
"It is NOT RECOMMENDED that implementations permit the use of "reject" 
with actions that cause mail delivery, such as "keep", "fileinto", and 
"redirect"."  But, we have kept the original restriction.


--
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: command line deletion of files

Re: command line deletion of files

Re: Twoskip DB files broken

Re: command line deletion of files

Re: watching and processing a Spam folder for each user

Re: watching and processing a Spam folder for each user

Re: watching and processing a Spam folder for each user

Re: watching and processing a Spam folder for each user

Re: watching and processing a Spam folder for each user

Re: command line deletion of files

Re: command line deletion of files

Re: watching and processing a Spam folder for each user

Re: command line deletion of files

watching and processing a Spam folder for each user

Re: command line deletion of files

Re: command line deletion of files

command line deletion of files

Twoskip DB files broken

Huge performance problems after updating from 2.4 to 2.5.9

Re: why sieve's "reject" is incompatible with imap4flags actions ?

20 matches

Site Navigation

Mail list logo

Footer information