Re: [Mimedefang] file descriptor scope and embedded perl
Apologies about last mail - been along weekend. > No; slaves are single-threaded in separate > processes. > Yes by using fork(2). I was thinking that dup(2) may have been copying the file descriptor [that I wasn't declaring/initialising properly int he filter script] between slave processes somewhere - but you also mention closing them all afterwards. I'll check my code again. > OK. (Some Canadians use *ise*, but I've become > Americanised ... uh ... Americanized.) ;) Thanks for your help. Chris __ Do you Yahoo!? Yahoo! Mail Address AutoComplete - You start. We finish. http://promotions.yahoo.com/new_mail ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] file descriptor scope and embedded perl
> No; slaves are single-threaded in separate > processes. > Yes by using fork(2). I was thinking that dup(2) may have been copying the file descriptor [that I was > > As I say - putting dummy connects within the > > filter_initialize (would have though it would be > *ise* > > in Canada too ;) ) solves the problem. > > OK. (Some Canadians use *ise*, but I've become > Americanised ... uh ... Americanized.) > > Regards, > > David. > ___ > Visit http://www.mimedefang.org and > http://www.canit.ca > MIMEDefang mailing list > [EMAIL PROTECTED] > http://lists.roaringpenguin.com/mailman/listinfo/mimedefang > __ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] file descriptor scope and embedded perl
On Sat, 10 Jul 2004, Chris Masters wrote: > It seems to happen when a slave is reaped after > processing n number of mails. But slaves are in separate processes, so they should not share file descriptors. > Could this be a thread concurrency issue? No; slaves are single-threaded in separate processes. > As I say - putting dummy connects within the > filter_initialize (would have though it would be *ise* > in Canada too ;) ) solves the problem. OK. (Some Canadians use *ise*, but I've become Americanised ... uh ... Americanized.) Regards, David. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] file descriptor scope and embedded perl
> Does it happen all the time, or only after you do > something (eg after > you do an "md-mx-ctrl reread")? > I don't use "md-mx-ctrl reread". It seems to happen when a slave is reaped after processing n number of mails. Somehow this must be affecting other slaves by closing their file descriptors [even though they are declared/initalised only within filter functions]. Could this be a thread concurrency issue? As I say - putting dummy connects within the filter_initialize (would have though it would be *ise* in Canada too ;) ) solves the problem. I'll try and do some debugging when I get some time. Chris __ Do you Yahoo!? Yahoo! Mail Address AutoComplete - You start. We finish. http://promotions.yahoo.com/new_mail ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] file descriptor scope and embedded perl
Solved by performing a dummy call to force data source
handle declaration/initialisation/connection at slave
startup:
sub filter_initialize()
{
#Declare Connection handles
undef $My_dbh;
undef $My_Ldap;
#dummy call to make initial connection
get_db_connection();
get_ldap_connection();
}
--- "David F. Skoll" <[EMAIL PROTECTED]> wrote:
> On Tue, 6 Jul 2004, Chris Masters wrote:
>
> > I still have the problem of bad file descriptors:
>
> What's the actual log message? Does the database
> server log anything
> funny?
>
> Regards,
>
> David.
> ___
> Visit http://www.mimedefang.org and
> http://www.canit.ca
> MIMEDefang mailing list
> [EMAIL PROTECTED]
>
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
>
__
Do you Yahoo!?
Yahoo! Mail - 50x more storage than other providers!
http://promotions.yahoo.com/new_mail
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] file descriptor scope and embedded perl
On Tue, 6 Jul 2004, Chris Masters wrote: > > > I still have the problem of bad file descriptors: It's a mystery to me. I can't see any reason for this behavior. Does it happen all the time, or only after you do something (eg after you do an "md-mx-ctrl reread")? Regards, David. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] file descriptor scope and embedded perl
> > I still have the problem of bad file descriptors: > > What's the actual log message? Does the database > server log anything > funny? The test_query functions are failing for both MySQL and LDAP connections. This started happening directly after upgrading from 2.39 t 2.43. Jul 6 15:55:16 filter mimedefang-multiplexor[28955]: Slave 29 stderr: DBD::mysql::st execute failed: Lost connection to MySQL server during query at /usr/local/bin/connection_func.pl line 49. Jul 6 15:55:16 filter mimedefang.pl[1627]: test_query error: DBD::mysql::st execute failed: Lost connection to MySQL server during query at /usr/local/bin/connection_func.pl line 49. Jul 6 15:56:47 filter mimedefang.pl[18073]: Reconnecting LDAP: ldap_test_query failure: I/O Error Bad file descriptor Thanks for any help on this, Chris __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] file descriptor scope and embedded perl
On Tue, 6 Jul 2004, Chris Masters wrote: > I still have the problem of bad file descriptors: What's the actual log message? Does the database server log anything funny? Regards, David. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] file descriptor scope and embedded perl
Thanks for your replies.
I still have the problem of bad file descriptors:
I *declare* my db handles in filter_initialize:
sub filter_initialize()
{
#Declare Connection handles
undef $My_dbh;
}
I obtain a valid open db handle using a function
get_db_connection (as per David's suggestion a long
while ago - thanks BTW):
sub get_db_connection()
{
if (defined($My_dbh) && test_query($My_dbh))
{
return $My_dbh;
}
undef $My_dbh;
$My_dbh = connect_db();
return $My_dbh;
}
This is called within filter_begin, filter and
filter_end.
I'm getting test_query failures dotted around the logs
- luckily the database connection is then
re-established :)
There is now no db
declaration/initialisation/query/code outside of any
filter function and as such all should take place
after the fork(2).
Where am I going wrong?
Cheers, Chrsi
--- "David F. Skoll" <[EMAIL PROTECTED]> wrote:
> On Mon, 5 Jul 2004, Chris Masters wrote:
>
> > Since upgrading to the latest MIMEDefang today I
> have
> > bad file descriptor errors - I assume this is an
> > embedded perl scope issue.
>
> Possibly.
>
> > 1) I do *not* use filer_initialise
>
> You should probably use it. Any files opened at
> global scope outside
> any function are opened once upon initialization.
> When a slave is
> started, mimedefang-multiplexor calls fork(2) and
> essentially dup's
> the file descriptors. However, the multiplexor
> child process (the one
> that's about to morph into a slave) then calls
> close(2) on all open
> file descriptors. Perl doesn't know about this---it
> thinks the file
> descriptors are still open---so when you try to use
> them, you get
> error messages.
>
> > Can/Should I declare *slave* globals in
> > filter_initialise?
>
> Yes. (And it's ...ize rather than ...ise, because
> I'm in North America
> rather than the UK. :-))
>
> Regards,
>
> David.
> ___
> Visit http://www.mimedefang.org and
> http://www.canit.ca
> MIMEDefang mailing list
> [EMAIL PROTECTED]
>
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
>
__
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
http://promotions.yahoo.com/new_mail
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] file descriptor scope and embedded perl
On Mon, 5 Jul 2004, Chris Masters wrote: > Since upgrading to the latest MIMEDefang today I have > bad file descriptor errors - I assume this is an > embedded perl scope issue. Possibly. > 1) I do *not* use filer_initialise You should probably use it. Any files opened at global scope outside any function are opened once upon initialization. When a slave is started, mimedefang-multiplexor calls fork(2) and essentially dup's the file descriptors. However, the multiplexor child process (the one that's about to morph into a slave) then calls close(2) on all open file descriptors. Perl doesn't know about this---it thinks the file descriptors are still open---so when you try to use them, you get error messages. > Can/Should I declare *slave* globals in > filter_initialise? Yes. (And it's ...ize rather than ...ise, because I'm in North America rather than the UK. :-)) Regards, David. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] file descriptor scope and embedded perl
Chris Masters wrote: Hi All, Since upgrading to the latest MIMEDefang today I have bad file descriptor errors - I assume this is an embedded perl scope issue. So, I currently do the following: 1) I do *not* use filer_initialise 2) The file descriptor are global and are declared *outside* of any function. 3) valid connections are made using a 'connection test function' within functions called within the 3 main filter functions. 4) file descriptors are closed in filter_cleanup I take it that I shouldn't be doing 2? Can/Should I declare *slave* globals in filter_initialise? Thanks for your help, Chris Chris, The documentation for mimedefang-filter specifically states that if you're using Embedded Perl, you *MUST* use filter_initialize to initilize variables (such as file descriptors) that need to be seen across slaves. with embedded perl, the (outside of any subroutine) global variables are only initialized once on initial startup, and not on a per-slave basis. check the mimedefang-filter man pages for more specific information. hope this helps. alan ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] file descriptor scope and embedded perl
Hi All, Since upgrading to the latest MIMEDefang today I have bad file descriptor errors - I assume this is an embedded perl scope issue. So, I currently do the following: 1) I do *not* use filer_initialise 2) The file descriptor are global and are declared *outside* of any function. 3) valid connections are made using a 'connection test function' within functions called within the 3 main filter functions. 4) file descriptors are closed in filter_cleanup I take it that I shouldn't be doing 2? Can/Should I declare *slave* globals in filter_initialise? Thanks for your help, Chris __ Do you Yahoo!? New and Improved Yahoo! Mail - 100MB free storage! http://promotions.yahoo.com/new_mail ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
