Re: [Boston.pm] should closing a file descriptor also close the accompanying file handle?
On Tue, Nov 02, 2010 at 09:59:12PM -0400, Uri Guttman wrote: why are you concerned about closing the DATA handle? it is internal to the program. actually it is the handle the perl binary uses to read the source file and it is left at the seek point where the DATA starts. Because, I understand it to be best practice to close all file handles, as per: http://www.webreference.com/perl/tutorial/9/2.html Besides closing any filehandles that might have been opened ... http://en.wikipedia.org/wiki/Daemon_%28computer_software%29 Closing all inherited open files at the time of execution that are left open by the parent process, ... http://www.enderunix.org/docs/eng/daemon.php Unneccesarry [sic] descriptors should be closed before fork() system call (so that they are not inherited) or close all open descriptors as soon as the child process starts running. I also have here in the office a copy of Steven's Unix Network programming Volume 1, 2nd edition. Chapter 12 (Daemon Processes and inetd SuperServer) describes daemonizing a process; his example code lib/daemon_init.c also pointedly closes all file handles; review that source here: http://www.kohala.com/start/unpv12e/unpv12e.tar.gz I'm maintaining a perl module (internally) that allows for convienent daemonizing; I wrote it to be as 'correct' as possible. Until yesterday, I didn't realize that DATA and END were magic file descriptors in perl. As such, it would be easy to alter my module to preserve them. If I can find reference to other magic descriptors, then I'll take them into account as well. well, perl keeps it open so it can be used by anyone reading from DATA. If it keeps it open (despite my closing the file descriptor), then why does the daemonized process no longer have access to all of it's contents? this list (as most other rightly do) filters out attachments. so you should just paste your code in the email instead. That does make sense; I'll re-post soonish... Thanks for the feedback... uri -- Uri Guttman -- u...@stemsystems.com http://www.sysarch.com -- - Perl Code Review , Architecture, Development, Training, Support -- - Gourmet Hot Cocoa Mix http://bestfriendscocoa.com - -- Brian Reichert reich...@numachi.com 55 Crystal Ave. #286 Derry NH 03038-1725 USA BSD admin/developer at large ___ Boston-pm mailing list Boston-pm@mail.pm.org http://mail.pm.org/mailman/listinfo/boston-pm
Re: [Boston.pm] should closing a file descriptor also close the accompanying file handle?
On Wed, Nov 3, 2010 at 8:42 AM, Brian Reichert reich...@numachi.com wrote: On Tue, Nov 02, 2010 at 09:59:12PM -0400, Uri Guttman wrote: why are you concerned about closing the DATA handle? it is internal to the program. actually it is the handle the perl binary uses to read the source file and it is left at the seek point where the DATA starts. Because, I understand it to be best practice to close all file handles, as per: http://www.webreference.com/perl/tutorial/9/2.html Besides closing any filehandles that might have been opened ... http://en.wikipedia.org/wiki/Daemon_%28computer_software%29 Closing all inherited open files at the time of execution that are left open by the parent process, ... http://www.enderunix.org/docs/eng/daemon.php Unneccesarry [sic] descriptors should be closed before fork() system call (so that they are not inherited) or close all open descriptors as soon as the child process starts running. I also have here in the office a copy of Steven's Unix Network programming Volume 1, 2nd edition. Chapter 12 (Daemon Processes and inetd SuperServer) describes daemonizing a process; his example code lib/daemon_init.c also pointedly closes all file handles; review that source here: http://www.kohala.com/start/unpv12e/unpv12e.tar.gz I'm maintaining a perl module (internally) that allows for convienent daemonizing; I wrote it to be as 'correct' as possible. What was wrong with the CPAN module Proc::Daemon? Until yesterday, I didn't realize that DATA and END were magic file descriptors in perl. As such, it would be easy to alter my module to preserve them. If I can find reference to other magic descriptors, then I'll take them into account as well. well, perl keeps it open so it can be used by anyone reading from DATA. If it keeps it open (despite my closing the file descriptor), then why does the daemonized process no longer have access to all of it's contents? this list (as most other rightly do) filters out attachments. so you should just paste your code in the email instead. That does make sense; I'll re-post soonish... Thanks for the feedback... uri -- Uri Guttman -- ...@stemsystems.com http://www.sysarch.com -- - Perl Code Review , Architecture, Development, Training, Support -- - Gourmet Hot Cocoa Mix http://bestfriendscocoa.com - -- Brian Reichert reich...@numachi.com 55 Crystal Ave. #286 Derry NH 03038-1725 USA BSD admin/developer at large ___ Boston-pm mailing list Boston-pm@mail.pm.org http://mail.pm.org/mailman/listinfo/boston-pm ___ Boston-pm mailing list Boston-pm@mail.pm.org http://mail.pm.org/mailman/listinfo/boston-pm
Re: [Boston.pm] should closing a file descriptor also close the accompanying file handle?
On Wed, Nov 03, 2010 at 09:04:40AM -0700, Ben Tilly wrote: What was wrong with the CPAN module Proc::Daemon? Nothing; I have to admit, I didn't know it was there, until I was largely done with what I had done. The internal module did not have the original function of daemonizing, and had been and over time, taking on many/most of the attributes of daemonizing. As such, I never explored the feasibility of refactoring to utilize Proc::Daemon. I haven't tried Proc::Daemon to see, but from it's write-up: http://search.cpan.org/~deti/Proc-Daemon-0.05/lib/Proc/Daemon.pod 6. The second child closes all open file descriptors. it may have the same issue with DATA and END... -- Brian Reichert reich...@numachi.com 55 Crystal Ave. #286 Derry NH 03038-1725 USA BSD admin/developer at large ___ Boston-pm mailing list Boston-pm@mail.pm.org http://mail.pm.org/mailman/listinfo/boston-pm
Re: [Boston.pm] should closing a file descriptor also close the accompanying file handle?
On Tue, Nov 02, 2010 at 09:59:12PM -0400, Uri Guttman wrote:
well, perl keeps it open so it can be used by anyone reading from
DATA. you can check for it specially with some code i got from
someone. it is special in that it is NOT tainted while all other handles
are.
Without delving further into whether or not it was appropriate for
me to close the descriptor for DATA, I'm still curious about what
I had originally written about: should closing a descriptor have
caused the associated file handle to close?
It was pointed out that this list strips attachments, so here is
my code.
I had described my symptoms in my original post, if any of this is
still of any interest to this list.
8--- cut here
#!/usr/bin/perl
# env PERL5LIB=. ./bar.pl
use strict;
use warnings;
use Foo;
# works as expected
Foo::info;
Foo::print_end_data;
Foo::close_fds;
#Foo::seek_data(); # insufficient
Foo::_init_DATA; # sufficient
Foo::info;
Foo::print_end_data;
8--- cut here
package Foo;
use strict;
use warnings;
use Sys::Syslog;
use POSIX qw(:limits_h);
use Cwd qw(abs_path);
my $_DATA_src=abs_path(__FILE__);
my $_DATA_pos=tell(DATA);
my @_DATA_stat=stat(DATA);
sub out_syslog
{
openlog $0, 'cons,pid', 'user';
my @str = map { defined $_ ? $_ : 'undefined' } @_;
syslog 'info', '%s', [. join(][, @str) . ];
closelog;
}
sub _init_DATA
{
if ( ! stat(DATA) )
{
open(DATA, '', $_DATA_src ); # ignore error?
}
seek DATA, $_DATA_pos, 0;
}
sub info
{
my $res;
# out_syslog (info, _DATA_stat, join(',', @_DATA_stat) );
# out_syslog (info, _DATA_pos, $_DATA_pos );
out_syslog (info, fileno, $res = fileno(DATA), $res );
out_syslog (info, tell, $res = tell(DATA), $res );
out_syslog (info, eof, $res = eof(DATA), $res );
my @stat = stat(DATA);
out_syslog (info, stat, join(',', @stat), $#stat );
}
sub close_fds
{
out_syslog (close_fds);
foreach my $fd (0 .. POSIX::sysconf(POSIX::_SC_OPEN_MAX))
{
my $res = POSIX::close $fd;
}
}
sub seek_data
{
my $pos = shift;
my $new = defined $pos ? $pos : $_DATA_pos ;
out_syslog (seek_data, $new);
seek DATA, $pos, 0;
}
sub print_end_data
{
my $line;
my $pos = tell(DATA);
while(DATA) { $line = $_; }
out_syslog (print_end_data, $line );
}
1;
__DATA__
Random text; don't mind me...
8--- cut here
--
Brian Reichert reich...@numachi.com
55 Crystal Ave. #286
Derry NH 03038-1725 USA BSD admin/developer at large
___
Boston-pm mailing list
Boston-pm@mail.pm.org
http://mail.pm.org/mailman/listinfo/boston-pm
[Boston.pm] should closing a file descriptor also close the accompanying file handle?
Perhaps this all just illustrates a pile of bad assumptions I've been making, but: I have a perl app that daemonizes itself. Among it's daemonizing efforts, it closes all file handles. My app currently utilizes a third-party perl module, which in turn makes use of the DATA file handle, as described in perldata(1). (For all my years of using perl, this was the first I ran into this mechanism, so it took a while to track down why this perl module was misbehaving.) My efforts to come up with a workaround were thwarted by a difficulty on my part in determining that the DATA file handle was was compromised. Attached is a simple perl module, and a test program that fires it's various methods; it scribble its output to syslog, if you want to follow along. The upshot is, when I used POSIX::close to close all file descriptors, somehow the DATA descriptor remained alive: - It was defined. - I could retrieve a file descriptor via fileno(). - I could use seek() and tell() to get/set my filepos. Only stat() seemed to report that there was something was amiss. I would have though fileno(), etc, would have also reported problems. Should they have? I admit, I've not pawed though and bug lists or change lists to see if this is different under newer releases... Example output: An initial working pass: Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [info][fileno][4][4] Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [info][tell][1333][1333] Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [info][eof][][] Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [info][stat][2054,32313,33152,1,0,0,0,1364,1288734213,1288734093,1288734093,4096,8][12] Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [print_end_data][Random text; don't mind me... ] Then, I close all descriptors, but can still utilize the file handle, except that I can't read past whatever my filepos was prior to the descriptor closing: Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [close_fds] Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [seek_data][1333] Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [info][fileno][4][4] Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [info][tell][1364][1364] Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [info][eof][1][1] Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [info][stat][][-1] Nov 2 21:43:33 10-160-62-45 ./bar.pl[5463]: [print_end_data][undefined] (In this pass the eof() flag is set, because the print_end_data() method was fired.) In my test code, if you comment out my use of seek_data(), and then use _init_DATA, you can see my use of stat() detects this case, and resets everything. My environment: # perl -v This is perl, v5.8.8 built for i386-linux-thread-multi # cat /etc/redhat-release CentOS release 5.4 (Final) -- Brian Reichert reich...@numachi.com 55 Crystal Ave. #286 Derry NH 03038-1725 USA BSD admin/developer at large ___ Boston-pm mailing list Boston-pm@mail.pm.org http://mail.pm.org/mailman/listinfo/boston-pm
Re: [Boston.pm] should closing a file descriptor also close the accompanying file handle?
BR == Brian Reichert reich...@numachi.com writes: BR I have a perl app that daemonizes itself. Among it's daemonizing BR efforts, it closes all file handles. BR My app currently utilizes a third-party perl module, which in turn BR makes use of the DATA file handle, as described in perldata(1). BR (For all my years of using perl, this was the first I ran into this BR mechanism, so it took a while to track down why this perl module BR was misbehaving.) why are you concerned about closing the DATA handle? it is internal to the program. actually it is the handle the perl binary uses to read the source file and it is left at the seek point where the DATA starts. BR The upshot is, when I used POSIX::close to close all file descriptors, BR somehow the DATA descriptor remained alive: BR - It was defined. BR - I could retrieve a file descriptor via fileno(). BR - I could use seek() and tell() to get/set my filepos. BR Only stat() seemed to report that there was something was amiss. well, perl keeps it open so it can be used by anyone reading from DATA. you can check for it specially with some code i got from someone. it is special in that it is NOT tainted while all other handles are. this list (as most other rightly do) filters out attachments. so you should just paste your code in the email instead. but again, my question is why do you want to close the DATA handle? it shouldn't affect daemonizing. in general, you only would need to close the handles attached to the controlling terminal (stdio) and possibly do a double fork. many daemons use handles to write log files, read config files, communicate with sockets, etc. uri -- Uri Guttman -- u...@stemsystems.com http://www.sysarch.com -- - Perl Code Review , Architecture, Development, Training, Support -- - Gourmet Hot Cocoa Mix http://bestfriendscocoa.com - ___ Boston-pm mailing list Boston-pm@mail.pm.org http://mail.pm.org/mailman/listinfo/boston-pm
