Re: consoliate mail archives with duplicate messages
On Monday 18 February 2008 10:43:46 am Russell L. Harris wrote: > Is there an archiving utility or other reasonably simple approach > whereby I can process these mail files to (1) considate them into a > single file or archive and (2) eliminate duplicate messages? KMail can do both, albeit slowly, in a drag-and-drop manner. -- Paul Johnson [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part.
Re: consoliate mail archives with duplicate messages
On Feb 19, 2008, at 3:53 AM, Johann Spies wrote: Over the years, every time I have reinstalled Debian, I have saved the mail files. Now I have a dozen or more files, with many duplicate messages. Is there an archiving utility or other reasonably simple approach whereby I can process these mail files to (1) considate them into a single file or archive and (2) eliminate duplicate messages? I would unpack those files into a Maildir and if you then have duplicate emails you can remove them with fdupes. That will find emails that are bit-for-bit identical. For emails that are the same except for some delivery headers, you may need to write a script that compares Message-ID: headers. Rick -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: consoliate mail archives with duplicate messages
* Jochen Schulz <[EMAIL PROTECTED]> [080219 08:49]: > Russell L. Harris: > > > > Over the years, every time I have reinstalled Debian, I have saved the > > mail files. Now I have a dozen or more files, with many duplicate > > messages. > > Mbox files? Some are mbox, from use of Eudora (back in the Window$ era) and Thunderbird and Balsa and Sylpheed. More recent files are mairdir, from gnus and mutt. RLH -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: consoliate mail archives with duplicate messages
Russell L. Harris: > > Over the years, every time I have reinstalled Debian, I have saved the > mail files. Now I have a dozen or more files, with many duplicate > messages. Mbox files? > Is there an archiving utility or other reasonably simple approach > whereby I can process these mail files to (1) considate them into a > single file or archive and (2) eliminate duplicate messages? If you have mbox files, you could just `cat` them into one file, open it with mutt and then press 'D' followed '~='. J. -- As a child I pulled the legs from a spider. [Agree] [Disagree] <http://www.slowlydownward.com/NODATA/data_enter2.html> signature.asc Description: Digital signature
Re: consoliate mail archives with duplicate messages
> Over the years, every time I have reinstalled Debian, I have saved the > mail files. Now I have a dozen or more files, with many duplicate > messages. > > Is there an archiving utility or other reasonably simple approach > whereby I can process these mail files to (1) considate them into a > single file or archive and (2) eliminate duplicate messages? I would unpack those files into a Maildir and if you then have duplicate emails you can remove them with fdupes. Regards Johann -- Johann Spies Telefoon: 021-808 4036 Informasietegnologie, Universiteit van Stellenbosch "And the angel answered and said unto the women, Fear not ye: for I know that ye seek Jesus, which was crucified. He is not here: for he is risen, as he said...Therefore be ye also ready; for in such an hour as ye think not the Son of man cometh." Matthew 28:5,6; 24:44 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
consoliate mail archives with duplicate messages
Over the years, every time I have reinstalled Debian, I have saved the mail files. Now I have a dozen or more files, with many duplicate messages. Is there an archiving utility or other reasonably simple approach whereby I can process these mail files to (1) considate them into a single file or archive and (2) eliminate duplicate messages? RLH -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Remove duplicate messages plug-in for Icedove
Has anyone figured out how to get this plug-in working for Icedove in Debian Etch? The version of Icedove on my machine is: 1.5.0.14pre. https://addons.mozilla.org/en-US/thunderbird/addon/956 Thanks in advance for any ideas. Keith -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Duplicate messages
I too get duplicates all the time. Using Sylpheed for email client. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Duplicate messages
Jonathan Opperman wrote: On Friday 19 March 2004 08:46, Johann Spies wrote: Am I the only one who receives nearly all the messages from this mailing list in in duplicate for the past weeks? This morning I have deleted 138 duplicates. Regards Johann [...] I'm not receiving any duplicates (-: . There can be only one (-: . And none here. Take your client to therapy, it's schitzophrenic. Regards, David. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Duplicate messages
On Fri, Mar 19, 2004 at 09:41:18AM +0100, Werner Mahr wrote: > > Are they for the same address? It seems to be not. I was convinced that I did unsubscribe one address more than a year ago ... Anyhow, thanks for your question. Now I know how to solve the problem. Regards Johann -- Johann Spies Telefoon: 021-808 4036 Informasietegnologie, Universiteit van Stellenbosch " And God shall wipe away all tears from their eyes; and there shall be no more death, neither sorrow, nor crying, neither shall there be any more pain: for the former things are passed away." Revelations 21:4 signature.asc Description: Digital signature
Re: Duplicate messages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Am Freitag, 19. März 2004 07:46 schrieb Johann Spies: > Am I the only one who receives nearly all the messages from this > mailing list in in duplicate for the past weeks? > > This morning I have deleted 138 duplicates. Are they for the same address? - -- MfG usw. Werner Mahr registered Linuxuser: 295882 -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFAWrIx1jkT71DQrmARAhO4AJ9GAWJL21fiqGDsXeocI2MokX+34wCeLE3b A3KKbL6NlyPYhUdaYAiB5Ow= =N3Nr -END PGP SIGNATURE-
Re: Duplicate messages
Incoming from Johann Spies: > Am I the only one who receives nearly all the messages from this > mailing list in in duplicate for the past weeks? I don't know about that, but I for one am not receiving duplicates. -- Any technology distinguishable from magic is insufficiently advanced. (*) http://www.spots.ab.ca/~keeling - - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Duplicate messages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jonathan Opperman <[EMAIL PROTECTED]> writes: > On Friday 19 March 2004 08:46, Johann Spies wrote: >> Am I the only one who receives nearly all the messages from this >> mailing list in in duplicate for the past weeks? >> >> This morning I have deleted 138 duplicates. >> >> Regards >> Johann > [...] > > I'm not receiving any duplicates (-: . There can be only one (-: . Yeah, the procmail HOWTO even has an example that will filter out duplicates. - -- .''`. Paul Johnson <[EMAIL PROTECTED]> : :' : `. `'` proud Debian admin and user `- Debian. Because it *must* work. debian.org aboutdebian.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFAWpq7UzgNqloQMwcRAs2dAJ9Db8c1K09AO2MKC8hIVIQjYf66GQCgp93G GJ7AkSSdQ1VchHGoaZt+Uqc= =RMOR -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Duplicate messages
On Friday 19 March 2004 08:46, Johann Spies wrote: > Am I the only one who receives nearly all the messages from this > mailing list in in duplicate for the past weeks? > > This morning I have deleted 138 duplicates. > > Regards > Johann [...] I'm not receiving any duplicates (-: . There can be only one (-: . Ciao Jono -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Duplicate messages
Am I the only one who receives nearly all the messages from this mailing list in in duplicate for the past weeks? This morning I have deleted 138 duplicates. Regards Johann -- Johann Spies Telefoon: 021-808 4036 Informasietegnologie, Universiteit van Stellenbosch " And God shall wipe away all tears from their eyes; and there shall be no more death, neither sorrow, nor crying, neither shall there be any more pain: for the former things are passed away." Revelations 21:4 signature.asc Description: Digital signature
Re: regarding duplicate messages
On Thu, 17 Jan 2002, dman wrote: > Regarding duplicate messages on debian-user, here is the relevant > portion of the headers from a recent one : > > > Received: from murphy.debian.org ([216.234.231.6]) by ritvax.isc.rit.edu > (PMDF V5.2-32 #40294) with SMTP id <[EMAIL PROTECTED]> > for [EMAIL PROTECTED] (ORCPT rfc822;[EMAIL PROTECTED]); > Wed, 16 Jan 2002 15:25:34 -0500 (EST) > Received: (qmail 6608 invoked by uid 38); Wed, 16 Jan 2002 20:13:00 + > Received: (qmail 6118 invoked from network); Wed, 16 Jan 2002 20:12:51 + > Received: from unknown (HELO imail.imaginet.co.za) (196.15.145.6) by > murphy.debian.org with SMTP; Wed, 16 Jan 2002 20:12:51 + > > > > Received: from murphy.debian.org ([216.234.231.6]) by ritvax.isc.rit.edu > (PMDF V5.2-32 #40294) with SMTP id <[EMAIL PROTECTED]> > for [EMAIL PROTECTED] (ORCPT rfc822;[EMAIL PROTECTED]); > Wed, 16 Jan 2002 15:57:10 -0500 (EST) > Received: (qmail 6608 invoked by uid 38); Wed, 16 Jan 2002 20:13:00 + > Received: (qmail 6118 invoked from network); Wed, 16 Jan 2002 20:12:51 + > Received: from unknown (HELO imail.imaginet.co.za) (196.15.145.6) by > murphy.debian.org with SMTP; Wed, 16 Jan 2002 20:12:51 + > > > Everything is identical up to the point that murphy.d.o transfers the > message to my school's MX handler. It seems that murphy is the one > causing the duplicates. Too bad qmail doesn't provide more > information. All the information is there. In smtp, duplicates are possible, if other end cuts the connection early without confirming the transfer. Usually this happens with hosts behind slow connections. -- "And if the messenger would shoot first?"
regarding duplicate messages
Regarding duplicate messages on debian-user, here is the relevant portion of the headers from a recent one : Received: from murphy.debian.org ([216.234.231.6]) by ritvax.isc.rit.edu (PMDF V5.2-32 #40294) with SMTP id <[EMAIL PROTECTED]> for [EMAIL PROTECTED] (ORCPT rfc822;[EMAIL PROTECTED]); Wed, 16 Jan 2002 15:25:34 -0500 (EST) Received: (qmail 6608 invoked by uid 38); Wed, 16 Jan 2002 20:13:00 + Received: (qmail 6118 invoked from network); Wed, 16 Jan 2002 20:12:51 + Received: from unknown (HELO imail.imaginet.co.za) (196.15.145.6) by murphy.debian.org with SMTP; Wed, 16 Jan 2002 20:12:51 + Received: from murphy.debian.org ([216.234.231.6]) by ritvax.isc.rit.edu (PMDF V5.2-32 #40294) with SMTP id <[EMAIL PROTECTED]> for [EMAIL PROTECTED] (ORCPT rfc822;[EMAIL PROTECTED]); Wed, 16 Jan 2002 15:57:10 -0500 (EST) Received: (qmail 6608 invoked by uid 38); Wed, 16 Jan 2002 20:13:00 + Received: (qmail 6118 invoked from network); Wed, 16 Jan 2002 20:12:51 + Received: from unknown (HELO imail.imaginet.co.za) (196.15.145.6) by murphy.debian.org with SMTP; Wed, 16 Jan 2002 20:12:51 + Everything is identical up to the point that murphy.d.o transfers the message to my school's MX handler. It seems that murphy is the one causing the duplicates. Too bad qmail doesn't provide more information. -D -- Trust in the Lord with all your heart and lean not on your own understanding; in all your ways acknowledge Him, and He will make your paths straight. Proverbs 3:5-6
Re: Deleting duplicate messages with formail/procmail
* Jason Rashaad Jackson <[EMAIL PROTECTED]> spake thus: > > I'm not sure about what you are doing with formail currently, but I have the > > following at the top of my .procmailrc script and it works great: > > > > # avoid duplicate messages > > :0 Whc: msgid.lock > > | formail -D 16384 Admin/msgid.cache > > > > :0 a: > > Admin/duplicates > > > > Duplicates get dumped into the file duplicates in ~/Main/Admin just in case > > I > > That looks good for avoiding future duplicates, but I need something that I > can run from the command-line a la 'cat mbox | formail -s procmail' that > will delete any duplicates it finds and leave the originals intact. Any > thoughts? Yeah. $ mv mbox mbox-old && formail -s procmail < mbox-old Your duplicate-free mail should now be in mbox. Regards, Stig -- brautaset.org Registered Linux User 107343
Re: Deleting duplicate messages with formail/procmail
> I'm not sure about what you are doing with formail currently, but I have the > following at the top of my .procmailrc script and it works great: > > # avoid duplicate messages > :0 Whc: msgid.lock > | formail -D 16384 Admin/msgid.cache > > :0 a: > Admin/duplicates > > Duplicates get dumped into the file duplicates in ~/Main/Admin just in case I That looks good for avoiding future duplicates, but I need something that I can run from the command-line a la 'cat mbox | formail -s procmail' that will delete any duplicates it finds and leave the originals intact. Any thoughts? -- my card... ___ | | |Jason Rashaad Jackson | | | | |\ /| UNIX Systems Administrator | | | | 2032 Dana Building | | | |\ /| | (Office) 734.615.1422 | | | | . | | (Mobile) 734.649.6641 | | |___| |___| http://www.umich.edu/~jrashaad | | G O B L U E | |___| pgpmQ5rJoHW6g.pgp Description: PGP signature
Re: Deleting duplicate messages with formail/procmail
On Mon, Sep 24, 2001 at 11:02:31AM -0400, Jason Rashaad Jackson wrote: > OK. Due to some ignorance on my part (errors with my procmail filters as > well as using 'formail -s procmail' with extreme prejudice) I now have many > many MANY copies of individual messages in all my mail folders. Is there a > way that I can run a filter to delete all but the original (or just all > copies) message out of a specific folder? I tried including dupcheck.dc > from the procmail-lib package, then running formail -s procmail with the > source as the original folder and the output to a new folder, but I wound up > with the same info in both folders. As always, any help offered will be > much appreciated. I'm not sure about what you are doing with formail currently, but I have the following at the top of my .procmailrc script and it works great: # avoid duplicate messages :0 Whc: msgid.lock | formail -D 16384 Admin/msgid.cache :0 a: Admin/duplicates Duplicates get dumped into the file duplicates in ~/Main/Admin just in case I want to see them. A nightly cron job removes them by moving that mail file to duplicates.old. I would go through your procmail script and make sure that it isn't creating duplicates when it shouldn't be. I would go light on the 'c' flag... it should only be used when you want a duplicate made. -- John Patton [EMAIL PROTECTED] "The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt." - Bertrand Russell
Deleting duplicate messages with formail/procmail
OK. Due to some ignorance on my part (errors with my procmail filters as well as using 'formail -s procmail' with extreme prejudice) I now have many many MANY copies of individual messages in all my mail folders. Is there a way that I can run a filter to delete all but the original (or just all copies) message out of a specific folder? I tried including dupcheck.dc from the procmail-lib package, then running formail -s procmail with the source as the original folder and the output to a new folder, but I wound up with the same info in both folders. As always, any help offered will be much appreciated. -- my card... ___ | | |Jason Rashaad Jackson | | | | |\ /| UNIX Systems Administrator | | | | 2032 Dana Building | | | |\ /| | (Office) 734.615.1422 | | | | . | | (Mobile) 734.649.6641 | | |___| |___| http://www.umich.edu/~jrashaad | | G O B L U E | |___| pgpuBiwi5Afwe.pgp Description: PGP signature
exim/filter: zapping duplicate messages?
On Wed, Jun 06, 2001 at 11:55:56AM -0300, John R Lenton wrote: > On Wed, Jun 06, 2001 at 05:41:58AM -0500, will trillich wrote: > > 3) anybody got some high-power, dual-exhaust, chrome-plated, > > twin overhead cam examples of exim filters to get us > > .forwarding newbies off the ground? i've seen the simple ones > > they've got in /usr/share/doc/exim/filter.html ... > > my experience is that that attitude is due to procmail. With the > exim filters your power consumption is lower, so you rarely need > dual exahust, and chrome only if you like it. how does exim/filter go about purging cloned messages (both to me indivindually and to a mailing list, both of which arrive in my inbox)? or do we still fall back to "formail -D"? -- DEBIAN NEWBIE TIP #17 from Will Trillich <[EMAIL PROTECTED]> : Why is TELNET considered spawn of Satan? Because every keystroke sent by you, is transmitted -- clear as a bell -- across the ether. Anyone can sniff your network packets and see what you're typing -- passwords, love letters, cold fusion blueprints... It's as secure as shouting from the rooftops. Everyone can see! Also see http://newbieDoc.sourceForge.net/ ...
Re: exim/filter: zapping duplicate messages?
On Wed, Jun 06, 2001 at 03:33:43PM -0500, will trillich wrote: > On Wed, Jun 06, 2001 at 11:55:56AM -0300, John R Lenton wrote: > > On Wed, Jun 06, 2001 at 05:41:58AM -0500, will trillich wrote: > > > 3) anybody got some high-power, dual-exhaust, chrome-plated, > > > twin overhead cam examples of exim filters to get us > > > .forwarding newbies off the ground? i've seen the simple ones > > > they've got in /usr/share/doc/exim/filter.html ... > > > > my experience is that that attitude is due to procmail. With the > > exim filters your power consumption is lower, so you rarely need > > dual exahust, and chrome only if you like it. > > how does exim/filter go about purging cloned messages (both to > me indivindually and to a mailing list, both of which arrive in > my inbox)? or do we still fall back to "formail -D"? I filter the mails from lists into list-specific mailboxes, and leave emails to me in the inbox. That way the ones to me get my immediate attention, but I don't loose the thread. I'm pondering the other one :) -- John Lenton ([EMAIL PROTECTED]) -- Random fortune: Para Dios todo es hermoso, bueno y justo. Los hombres han concebido lo justo y lo injusto. -- Heráclito de Efeso. (535-475 A.C.) Filósofo griego. pgpyEzhxcqBI0.pgp Description: PGP signature
Re: exim/filter: zapping duplicate messages?
On Thu, Jun 07, 2001 at 12:13:04AM -0300, John R Lenton wrote: > On Wed, Jun 06, 2001 at 03:33:43PM -0500, will trillich wrote: > > how does exim/filter go about purging cloned messages (both to > > me indivindually and to a mailing list, both of which arrive in > > my inbox)? or do we still fall back to "formail -D"? > > I filter the mails from lists into list-specific mailboxes, and > leave emails to me in the inbox. That way the ones to me get my > immediate attention, but I don't loose the thread. when checking something like # Exim filter if $h_to: contains "[EMAIL PROTECTED]" then save Mail/personal endif apparently it treats the "To:" field as one big string: To: [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED] so if a list-reply wound up looking like To: debian-user@lists.debian.org, [EMAIL PROTECTED] i'd have to find some other way to figure out if it arrived at my doorstep via debian-user broadcast, or by personal missive. how? -- DEBIAN NEWBIE TIP #1 from Will Trillich <[EMAIL PROTECTED]> : Looking to use your Debian machine as a FIREWALL? No problem! Try "apt-get install ipmasq"... After you've got your /etc/network/interfaces file set up properly, ipmasq will save you lots of work, setting up firewall and routing tables automatically. Also see http://newbieDoc.sourceForge.net/ ...
Re: Duplicate Messages II
On Thu, Jul 09, 1998 at 08:01:59PM -0400, Somnolent wrote: > Well still no response from the people at the Fidonet gateway, nor from > their ISP. > > So figuring I'd get faster service from their upstream provider I > e-mailed them ([EMAIL PROTECTED], [EMAIL PROTECTED]). Both came > back 550 - user unknown. > > > > So it looks like no one in Italy is going to stop the duplicate > messages. Is it RBL time yet? :) `postmaster' is the standard -- abuse is a relatively new one due to current UCE/spam problems, and root is a unixism. Hamish -- Hamish Moffatt, [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED] Latest Debian packages at ftp://ftp.rising.com.au/pub/hamish. PGP#EFA6B9D5 CCs of replies from mailing lists are welcome. http://hamish.home.ml.org -- Unsubscribe? mail -s unsubscribe [EMAIL PROTECTED] < /dev/null
Duplicate Messages II
Well still no response from the people at the Fidonet gateway, nor from their ISP. So figuring I'd get faster service from their upstream provider I e-mailed them ([EMAIL PROTECTED], [EMAIL PROTECTED]). Both came back 550 - user unknown. So it looks like no one in Italy is going to stop the duplicate messages. Is it RBL time yet? :) -- Unsubscribe? mail -s unsubscribe [EMAIL PROTECTED] < /dev/null
Duplicate Messages
[EMAIL PROTECTED] (linux.debian.user) Whoever this is needs to do a serious adjustment. They're duping messages and beying listed in both the TO: and FROM: fields. All the duplicate messages have come from this address... :( (I'm no mail GOD so I dunno whats wrong, just a wee bit annoying!) debian-user@lists.debian.org -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word "unsubscribe" to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
recent duplicate messages -- My Fault
The recent duplicate messages that appeared on debian-user and debian-devel were my fault. An error in my procmail script was resending things out, and I didnt catch it until several messages slipped out. I guess I should have tested it better before I unlocked the mail queue. Please forgive me, Erv Walter -- Graduate Student[EMAIL PROTECTED] Department of Chemistry [EMAIL PROTECTED] Univ of Wisconsin-Madison [EMAIL PROTECTED] -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word "unsubscribe" to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Duplicate Messages
Help! I'm drowning in duplicate messages. Is there a problem with the list server? I've been getting up to four or five duplicates of the same message. Victor -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word "unsubscribe" to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED]
