Re: Favorite email parser?

[Brian Gerard]

Forgot to mention the platform.  This will be on a FreeBSD system, and
the emails will be in the standard flat file format (mbox).  Didn't want
anyone going too deep on the M$/Exchange side of the house.  :)


  /~~\
 | Brian GerardRed meat isn't bad for you. Fuzzy  |
 | First initial + 'lists'  blue-green meat is bad for you.   |
 | at technobrat dot com  |
  \__/

-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: Favorite email parser?

[Daniel Staal]

--As of Tuesday, June 29, 2004 3:46 PM -0700, Brian Gerard is alleged to 
have said:

1) Does this sound like a reasonable approach to the problem, given the
relatively low amount of detail I've provided?  Anyone dealt with this
type of thing before and have any gotchas for me?
I can't say anything about gotchas, but I would look at trying to eliminate 
the mail step.  It might be possible to directly query the resource 
monitors, or modify them to do the database entry instead of mailing.

Of course, that depends on what you are doing.  If you have a standard 
'alert format' that everything uses, and mails to the same address, from 
all over a large site/company, where every system has it's own alert 
generator, this may be the best way.  Just saying to check if they are all 
a couple line Perl scripts, then it may be you can create a database entry 
template and stick it in instead of the email code.

2) From what I've read, Postgres beats MySQL performance-wise, which is
why I chose it, but IANADBA.  Any votes on a free DB to use for this?
Obviously, it needs to have a perl interface or CPAN module available.
Regardless of which DB you like, what's your favorite module for
interacting with it?  ...and, umm, if you could point me to any "Getting
Started" docs, that would be just lovely.  <:)
The DBI modules are what you want to use here.  If you use them correctly 
it almost doesn't matter what database you are using (at least to the 
programmer).

I'd recommend the book 'Programming for the Perl DBI' as an intro.  It goes 
over everything you're likely to need.  (O'Reilly press.)

As for DB...  It would depend on what you need out of it.  A little 
googling will bring up sites with the advantages and disadvantages of the 
various DBs for you, with some explanation of when you should care about 
each.

3) Which module would you suggest for parsing email?  There are a few of
them out there, and I really don't have the time to try each one out
until I find the one I like the best.  :)
I've been just doing some research into this for a project I'm working on, 
and have been impressed by the Mail::Box module set.  Of course, ask me in 
a couple weeks and my answer may change...  But generally that module set 
looks to be able to do anything you would want to do with email.

Daniel T. Staal
---
This email copyright the author.  Unless otherwise noted, you
are expressly allowed to retransmit, quote, or otherwise use
the contents for non-commercial purposes.  This copyright will
expire 5 years after the author's death, or in 30 years,
whichever is longer, unless such a period is in excess of
local copyright law.
---
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: Favorite email parser?

[JupiterHost.Net]

Brian Gerard wrote:
Hi all-
Hello,
I'm about to start a project that goes a little something like this...
Right now, alerts get automatically emailed to a central address which due
to the volume is largely ignored.  You know the drill: "Ooo! It would be
great if we got notified when resource X is running low!  Then we could
anticipate the need for it and get ahead of the game."  Repeat for resources
A-Z, AA-ZZ, etc, etc and you get a mailbox that quickly becomes filled with
alerts that no one reads, largely due to the fact that the reading alone
would be a full time job.  We can receive anywhere from a couple of hundred
to a couple of thousand of these per day.
So I'm planning to set up the address to forward to a backline server, where
my Brilliantly Written(tm) perl script will crunch said alerts.  I'd like it
to chop them up into little bits and insert the bits into an
intelligently-structured database of some sort.  I'm planning on Postgres at
this point.
Then there will be another script, available to the proper people via an
internal website, where queries can be made against the database according to
a yet-to-be-designed web interface.  The main thing I want is a simplification
of the alert viewing.  So you might see "3245 instances of alert Y", which
could be drilled down into based on various criteria, rather than the first
page (out of 65) of 50 alerts all of the same type.  Statistical trending and
other such pie-chart-able things may show up in the future, but not for now.
So my questions are these:
1) Does this sound like a reasonable approach to the problem, given the
relatively low amount of detail I've provided?  Anyone dealt with this type of
thing before and have any gotchas for me?

2) From what I've read, Postgres beats MySQL performance-wise, which is why I
Where'd you read that, postgres.org ? :)

chose it, but IANADBA.  Any votes on a free DB to use for this?  Obviously, it
MySQL is my .02 :) its always been there for me and soem quite hefty 
assignments..

needs to have a perl interface or CPAN module available.  Regardless of which
DB you like, what's your favorite module for interacting with it?  ...and,
umm, if you could point me to any "Getting Started" docs, that would be just
lovely.  <:)
3) Which module would you suggest for parsing email?  There are a few of them
out there, and I really don't have the time to try each one out until I find
the one I like the best.  :)
I'm a little bit away from a module that has an nice email parsing function.
It uses Mail::Internet underneath but take its a step further by 
simplifying the calling and returning other useful values like a plain 
text version and an attachement-less version (text or multipart with HTML)

It works like this:
 my %email = emailhash(\*STDIN);
 $email{FROM}
 $email{SUBJECT}
etc..
The only thing left is to parse the attachements into a nice hashref 
that can be used with the internal mailing function.

That might be for version 2 :) Other than that its quite handy..
Watch for SimpleMood.pm :) Odd name, easy to use and handy (I hope ;p)
Answers to any or all of the above gratefully received!
HTH
Lee.M - JupiterHost.Net
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: Favorite email parser?

[Randy W. Sims]

On 6/29/2004 6:46 PM, Brian Gerard wrote:
Hi all-
I'm about to start a project that goes a little something like this...
Right now, alerts get automatically emailed to a central address which due
to the volume is largely ignored.  You know the drill: "Ooo! It would be
great if we got notified when resource X is running low!  Then we could
anticipate the need for it and get ahead of the game."  Repeat for resources
A-Z, AA-ZZ, etc, etc and you get a mailbox that quickly becomes filled with
alerts that no one reads, largely due to the fact that the reading alone
would be a full time job.  We can receive anywhere from a couple of hundred
to a couple of thousand of these per day.
Uhmm, perhaps a better solution is an organizational one. Alerts lose 
there efficacy when there are an overload of them. Perhaps you should 
reconsider what constitutes an alert. However, if you're using the term 
alert as a synonym for a todo then it would make sense to develop a 
solution like you propose.

Just a thought,
Randy.

--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: Favorite email parser?

[Peter Scott]

In article <[EMAIL PROTECTED]>,
 [EMAIL PROTECTED] (Brian Gerard) writes:
>Right now, alerts get automatically emailed to a central address which due
>to the volume is largely ignored.  You know the drill: "Ooo! It would be
>great if we got notified when resource X is running low!  Then we could
>anticipate the need for it and get ahead of the game."  Repeat for resources
>A-Z, AA-ZZ, etc, etc and you get a mailbox that quickly becomes filled with
>alerts that no one reads, largely due to the fact that the reading alone
>would be a full time job.  We can receive anywhere from a couple of hundred
>to a couple of thousand of these per day.
>
>So I'm planning to set up the address to forward to a backline server, where
>my Brilliantly Written(tm) perl script will crunch said alerts.  I'd like it
>to chop them up into little bits and insert the bits into an
>intelligently-structured database of some sort.  I'm planning on Postgres at
>this point.
>
>Then there will be another script, available to the proper people via an
>internal website, where queries can be made against the database according to
>a yet-to-be-designed web interface.  The main thing I want is a simplification
>of the alert viewing.  So you might see "3245 instances of alert Y", which
>could be drilled down into based on various criteria, rather than the first
>page (out of 65) of 50 alerts all of the same type.  Statistical trending and
>other such pie-chart-able things may show up in the future, but not for now.
>
>So my questions are these:
>
>1) Does this sound like a reasonable approach to the problem, given the
>relatively low amount of detail I've provided? 

Yes.

>2) From what I've read, Postgres beats MySQL performance-wise, which is why I
>chose it, but IANADBA.  Any votes on a free DB to use for this?  

Which ever one you find easiest to set up.  If performance turns out to
be a problem, it's unlikely to make a difference which one you use.
Judge by other criteria.

>Obviously, it
>needs to have a perl interface or CPAN module available.  Regardless of which
>DB you like, what's your favorite module for interacting with it? 

DBI.  And it won't make any difference what database you use.

>3) Which module would you suggest for parsing email?  There are a few of them
>out there, and I really don't have the time to try each one out until I find
>the one I like the best.  :)

Check out Email::Simple.  Should be quite, er, simple.

-- 
Peter Scott
http://www.perldebugged.com/
*** NEW *** http://www.perlmedic.com/

-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: Favorite email parser?

[jeff rak]

::On Tuesday 29 June 2004 06:46 pm, Brian Gerard <[EMAIL PROTECTED]> 
wrote:
::Hi all-
::
::I'm about to start a project that goes a little something like this...
::
::Right now, alerts get automatically emailed to a central address which due
::to the volume is largely ignored.  You know the drill: "Ooo! It would be
::great if we got notified when resource X is running low!  Then we could
::anticipate the need for it and get ahead of the game."  Repeat for
:: resources A-Z, AA-ZZ, etc, etc and you get a mailbox that quickly becomes
:: filled with alerts that no one reads, largely due to the fact that the
:: reading alone would be a full time job.  We can receive anywhere from a
:: couple of hundred to a couple of thousand of these per day.
::

There's an open-source package out there called Nagios (www.nagios.org) which 
sounds like what your trying to accomplish. Nagios is written mostly in C 
(IIRC) and supports many different plugins (written in perl or C). 

-rak

-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: Favorite email parser?

[Wiggins d Anconia]

> --As of Tuesday, June 29, 2004 3:46 PM -0700, Brian Gerard is alleged to 
> have said:
> 



> 
> > 3) Which module would you suggest for parsing email?  There are a few of
> > them out there, and I really don't have the time to try each one out
> > until I find the one I like the best.  :)
> 
> I've been just doing some research into this for a project I'm working
on, 
> and have been impressed by the Mail::Box module set.  Of course, ask
me in 
> a couple weeks and my answer may change...  But generally that module set 
> looks to be able to do anything you would want to do with email.
> 

Others have covered the discussion well enough, I tend to concur with them.

Mail::Box is excellent and incredibly thorough, which also means it is
incredibly complex and has a pretty steep learning curve.  It was by far
the most robust mail handling I could find, and took me the longest to
learn.  It is also not the most efficient, but then that wasn't our goal.

Just my $.02 on MB,

http://danconia.org


-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

RE: Favorite email parser?

[NYIMI Jose (BMB)]

> -Original Message-
> From: Wiggins d Anconia [mailto:[EMAIL PROTECTED] 
> Sent: Wednesday, June 30, 2004 3:48 PM
> To: Perl Beginners
> Subject: Re: Favorite email parser?
> 

> Mail::Box is excellent and incredibly thorough, which also 
> means it is incredibly complex and has a pretty steep 
> learning curve.  It was by far the most robust mail handling 
> I could find, and took me the longest to learn.  It is also 
> not the most efficient, but then that wasn't our goal.

I'm using with a lot success Mail::Box and I'm curious to know
Which aspect of Mail::Box you found to be not effecient ?

Thanks,

José.

-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED] <http://learn.perl.org/> 
<http://learn.perl.org/first-response>




 DISCLAIMER 

"This e-mail and any attachment thereto may contain information which is confidential 
and/or protected by intellectual property rights and are intended for the sole use of 
the recipient(s) named above. 
Any use of the information contained herein (including, but not limited to, total or 
partial reproduction, communication or distribution in any form) by other persons than 
the designated recipient(s) is prohibited. 
If you have received this e-mail in error, please notify the sender either by 
telephone or by e-mail and delete the material from any computer".

Thank you for your cooperation.

For further information about Proximus mobile phone services please see our website at 
http://www.proximus.be or refer to any Proximus agent.


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>

RE: Favorite email parser?

[Wiggins d Anconia]

> 
> 
> > -Original Message-
> > From: Wiggins d Anconia [mailto:[EMAIL PROTECTED] 
> > Sent: Wednesday, June 30, 2004 3:48 PM
> > To: Perl Beginners
> > Subject: Re: Favorite email parser?
> > 
> 
> > Mail::Box is excellent and incredibly thorough, which also 
> > means it is incredibly complex and has a pretty steep 
> > learning curve.  It was by far the most robust mail handling 
> > I could find, and took me the longest to learn.  It is also 
> > not the most efficient, but then that wasn't our goal.
> 
> I'm using with a lot success Mail::Box and I'm curious to know
> Which aspect of Mail::Box you found to be not effecient ?
> 

Load time mostly, since it is a massive library. For our app that didn't
matter much since it runs for months at a time and we have much more
overhead with POE.  We are also running off a pretty old version, and
with all of the memory leak fixes and the C parser, it might be
significantly better than when I was doing load testing (year ago).

Doubt it would stack up compared to some of the other mail parsers, but
then they wouldn't in feature list come even close, but that is really a
hinderance of Perl in general rather than anything specific about MB. 
Essentially with the need for so many objects and doing everything
through methods there is a fair amount of overhead even if you just want
to do something trivial, but that *is* the point, it is a very good
interface for doing non-trivial things.

http://danconia.org

-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>

RE: Favorite email parser?

[NYIMI Jose (BMB)]

> -Original Message-
> From: Wiggins d Anconia [mailto:[EMAIL PROTECTED] 
> Sent: Wednesday, June 30, 2004 6:14 PM
> To: NYIMI Jose (BMB); Perl Beginners
> Subject: RE: Favorite email parser?
> 
> 
> > 
> > 
> > > -Original Message-
> > > From: Wiggins d Anconia [mailto:[EMAIL PROTECTED]
> > > Sent: Wednesday, June 30, 2004 3:48 PM
> > > To: Perl Beginners
> > > Subject: Re: Favorite email parser?
> > > 
> > 
> > > Mail::Box is excellent and incredibly thorough, which also
> > > means it is incredibly complex and has a pretty steep 
> > > learning curve.  It was by far the most robust mail handling 
> > > I could find, and took me the longest to learn.  It is also 
> > > not the most efficient, but then that wasn't our goal.
> > 
> > I'm using with a lot success Mail::Box and I'm curious to 
> know Which 
> > aspect of Mail::Box you found to be not effecient ?
> > 
> 
> Load time mostly, since it is a massive library. For our app 
> that didn't matter much since it runs for months at a time 
> and we have much more overhead with POE.  We are also running 
> off a pretty old version, and with all of the memory leak 
> fixes and the C parser, it might be significantly better than 
> when I was doing load testing (year ago).
> 
> Doubt it would stack up compared to some of the other mail 
> parsers, but then they wouldn't in feature list come even 
> close, but that is really a hinderance of Perl in general 
> rather than anything specific about MB. 
> Essentially with the need for so many objects and doing 
> everything through methods there is a fair amount of overhead 
> even if you just want to do something trivial, but that *is* 
> the point, it is a very good interface for doing non-trivial things.
> 

Thanks for sharing ...

José.


 DISCLAIMER 

"This e-mail and any attachment thereto may contain information which is confidential 
and/or protected by intellectual property rights and are intended for the sole use of 
the recipient(s) named above. 
Any use of the information contained herein (including, but not limited to, total or 
partial reproduction, communication or distribution in any form) by other persons than 
the designated recipient(s) is prohibited. 
If you have received this e-mail in error, please notify the sender either by 
telephone or by e-mail and delete the material from any computer".

Thank you for your cooperation.

For further information about Proximus mobile phone services please see our website at 
http://www.proximus.be or refer to any Proximus agent.


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>