Re: files checksum perl program help

[ram ram]

Thank you for the details.

the main task is that. On server1 , the master installation dir which contains 
around 100 shared library files (*.so)
the server2 also has same number of shared libraries. But in this installation  
there might be some shared libraries older than master install or newer than 
master install.
I need to figure out and report in a file that the library xxx_lib.so(server2) 
is different from Master(which is on server1).

Ramp




 From: Jim Gibson 
To: "beginners@perl.org"  
Sent: Thursday, January 5, 2012 11:34 AM
Subject: Re: files checksum perl program help
 
On 1/5/12 Thu  Jan 5, 2012  11:01 AM, "ram ram" 
scribbled:

> Thank You very much Jim for your immediate response.
> my requirement is that I have standard installation on server1 which is a
> master copy. I know the location of the installation directory which contains
> bunch of libraries.
> I have a another installation on Server2 for the same application. I do know
> the location of the installation dir.
> I have to compare all the libraries (checksums) on the server2 with the
> standard installation on server1, for all the files in the installation dir.
> I have to do dynamically compute the checksum of each file in the server2 and
> compare same with on server1.
> I am not sure which protocol suits best. right now I am considering NFS.
> Your help greatly appreciated.

(Please do not top post on this list. If you do not know what this means,
please ask or look at . Thanks.)


Using NFS (or SMB) will allow you to treat the files as if they are local.
Using SCP or FTP means that you will have to fetch the files first before
they can be compared.

Why do you need to compute checksums? If you are just trying to see if two
files are the same, you can compare them. For example, the Unix cmp program
will do that. The File::Compare module provides a Perl equivalent.

Checksums will be useful if generating and saving them allow you to avoid
comparing long files more than once. But if you are not in control of the
library files on the two servers, then you can't be sure a file has changed,
and you will have to read it again. Checksums would also be useful if you
had more than two servers to compare. You could compute the checksums once
and then make multiple comparisons of the various checksums instead of
comparing the files themselves.

However, if all you are doing is comparing the same file on two different
servers, computing the checksums doesn't buy you a lot. You still have to
read the whole file in order to compute a valid checksum.

Whatever scheme you use, your goal should be to only read each file once.

You might want to compare file sizes first. If two files are different
sizes, then they are not exact copies, and you do not need to read the
entire files. If two files are the same size, then they might be the same or
they might be different, and you will have to compare the files (or
checksums).

If you do use NFS, then the details of fetching the files will be handled by
the operating system. In that case, I would suggest the use of the
File::Find module for finding the files on the two servers.

How often do the files change? The answer may affect your choice of method.



-- 
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/

Re: files checksum perl program help

[Jim Gibson]

On 1/5/12 Thu  Jan 5, 2012  11:01 AM, "ram ram" 
scribbled:

> Thank You very much Jim for your immediate response.
> my requirement is that I have standard installation on server1 which is a
> master copy. I know the location of the installation directory which contains
> bunch of libraries.
> I have a another installation on Server2 for the same application. I do know
> the location of the installation dir.
> I have to compare all the libraries (checksums) on the server2 with the
> standard installation on server1, for all the files in the installation dir.
> I have to do dynamically compute the checksum of each file in the server2 and
> compare same with on server1.
> I am not sure which protocol suits best. right now I am considering NFS.
> Your help greatly appreciated.

(Please do not top post on this list. If you do not know what this means,
please ask or look at . Thanks.)


Using NFS (or SMB) will allow you to treat the files as if they are local.
Using SCP or FTP means that you will have to fetch the files first before
they can be compared.

Why do you need to compute checksums? If you are just trying to see if two
files are the same, you can compare them. For example, the Unix cmp program
will do that. The File::Compare module provides a Perl equivalent.

Checksums will be useful if generating and saving them allow you to avoid
comparing long files more than once. But if you are not in control of the
library files on the two servers, then you can't be sure a file has changed,
and you will have to read it again. Checksums would also be useful if you
had more than two servers to compare. You could compute the checksums once
and then make multiple comparisons of the various checksums instead of
comparing the files themselves.

However, if all you are doing is comparing the same file on two different
servers, computing the checksums doesn't buy you a lot. You still have to
read the whole file in order to compute a valid checksum.

Whatever scheme you use, your goal should be to only read each file once.

You might want to compare file sizes first. If two files are different
sizes, then they are not exact copies, and you do not need to read the
entire files. If two files are the same size, then they might be the same or
they might be different, and you will have to compare the files (or
checksums).

If you do use NFS, then the details of fetching the files will be handled by
the operating system. In that case, I would suggest the use of the
File::Find module for finding the files on the two servers.

How often do the files change? The answer may affect your choice of method.



-- 
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/

Re: files checksum perl program help

[ram ram]

Thank You very much Jim for your immediate response.
my requirement is that I have standard installation on server1 which is a 
master copy. I know the location of the installation directory which contains 
bunch of libraries.
I have a another installation on Server2 for the same application. I do know 
the location of the installation dir.
I have to compare all the libraries (checksums) on the server2 with the 
standard installation on server1, for all the files in the installation dir.
I have to do dynamically compute the checksum of each file in the server2 and 
compare same with on server1.
I am not sure which protocol suits best. right now I am considering NFS.
Your help greatly appreciated.

Regards,
Ramp




 From: Jim Gibson 
To: "beginners@perl.org"  
Sent: Wednesday, January 4, 2012 3:54 PM
Subject: Re: files checksum perl program help
 
On 1/4/12 Wed  Jan 4, 2012  3:38 PM, "ram ram" 
scribbled:

> Hi ,
>    Wish you a Very Happy and Wonderful New Year.
> I am a beginner in perl programming.
> 
> I request your help to write a perl program that takes 2 different directories
> on 2 different servers and find all the files in these 2 directories have the
> same files with same checksums.
> can anybody help me on this?

How are you going to access the files on the two servers? NFS? SMB? FTP?
SCP?

What kind of checksum do you want to use? Are the checksums already
computed, or do you need to compute them yourself?



--
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/

Re: Appending extra characters to file name

[John W. Krahn]

Alex Ardin wrote:

Hi,


Hello,


I frequently have to review logs that are captured with 'script' using
terminal but these logs have many control characters.

The intent of the script is the following:

1. Get a listing of all the logs in a directory, storing the list of
files in an array.


perldoc -f opendir
perldoc -f readdir
perldoc -f glob



2. For each filename in the array, create a new filename but append a
".x" to the original filename, so that file 'a' would result in a new
file 'a.x', as an example.
3. Open the original file, search and remove any control characters
(such as  ^M, ^C, and so on) line by line and store the results in the
newly created file ending in .x (such as the a.x example above).
4. After reaching the end file, close both files.
5. Move the newly created .x file so that it replaces the original
file (or 'mv a.x a' in the example above).

What happens is that the script works fine, all control characters are
replaced in the file, but the move fails, so that if I start with a
directory containing these files:

xong$ ls -rlt
-rw-r--r--  1 rootroot  staff  50033 Oct 11 17:43 xyz
-rw-r--r--  1 rootroot  staff  50033 Oct 13 18:57 123

when the script finishes executing, I end up with these file, the
first two are the original files with all the control characters, the
last two are the newly created files with the control characters
removed:

xong$ ls -rlt
-rw-r--r--  1 rootroot  staff  50033 Oct 11 17:43 xyz
-rw-r--r--  1 rootroot  staff  50033 Oct 13 18:57 123
-rw-r--r--  1 rootroot  staff  49792 Jan  4 22:14 xyz?.x
-rw-r--r--  1 rootroot  staff  49792 Jan  4 22:14 123?.x


The question mark after the file name means that you haven't removed the 
newline character.




Here's the errors I see when the script executes, the comments are
just for debugging:

Here's a listing of the files:
123
xyz

Here's newfile: 123
.xHere's oldfile: 123


There you can see the newline between '123' and '.x'



Now mv file
Here's newfile: 123
.xHere's oldfile: 123


And again there.



usage: mv [-f | -i | -n] [-v] source target
mv [-f | -i | -n] [-v] source ... directory
sh: line 1: .x: command not found


And the shell (sh) outputs an error message because of the newline.



I'm not sure why this is happening, the newfile should contain "123.x"
but instead the .x is appearing on the newline.  I've gone through the
"Programming Perl" book and can't seem to fine another way to do this,
and I've also searched this group, and didn't find anything similar.

Here's the script:

#!/usr/bin/perl

#
# Delete all control characters in a group of files in a directory.
#

#use strict;
use warnings;

if ($#ARGV != 0) {
 print "\nusage: get_file_list.pl directory\n\n";
 exit;
}

$dir_path = $ARGV[0];
chdir "$dir_path" or die "\ncd failed, check file path!\n\n";

$cmd = "pwd";
if(system($cmd)) { print "\n\npwd failed.\n"; }

$cmd = "ls -l";
if(system($cmd)) { print "\n\nFile listing failed.\n"; }

# Store file listing in a array:
$cmd = "ls";
my @fl = qx($cmd);


In list context (like you are using above) the qx// operator returns 
lines of text with each line ending with a newline.


You could use chomp() to remove the newlines or just use perl's built-in 
functions for reading directory entries.




print "\nHere's a listing of the files:\n";

foreach $file (@fl) {
 print $file;
}

I thought it was a result of my foreach loop but the loop exits after
the last file is reached, so there's nothing to force a newline when
my script copies the original filename to a new file name ending in
".x".

foreach $file (@fl) {
$oldfile = $file;
my $newfile = $oldfile . ".x";
print("Here's newfile: $newfile");
print("Here's oldfile: $oldfile");

open(OF, $oldfile);
open(NF, ">$newfile");


You should _always_ verify that open() worked correctly before trying to 
use a possibly invalid filehandle.




# read in each line of the file
while ($line =) {
$line =~ s/\cM//g;
$line =~ s/\cH//g;
$line =~ s/\cC//g;
$line =~ s/\cL//g;
$line =~ s/\cG//g;
$line =~ s/\cP//g;
$line =~ s/\cQ//g;
$line =~ s/\cR//g;
$line =~ s/\c[//g;
print NF $line;
}

print("\nNow mv file\n");
print("Here's newfile2: $newfile");
print("Here's oldfile2: $oldfile");

$cmd = "mv $newfile $oldfile";
if(system($cmd)) { print "rename failed\n"; }


You should probably just use perl's built-in rename() function.

perldoc -f rename



close(OF);
close(NF);
}

I also thought it had been the result of my closing my files after I
attempted the move, but I get the same results if I move these lines:

close(OF);
close(NF);

above this line:

print("\nNow mv file\n");

so that the files are closed and then attempt to move them.

Is it possible to

Re: Splitting and printing on a single line

[John W. Krahn]

bob wrote:

Hello


Hello,


I am very new to perl and I need some guidance on the following issue

My data set is like this

---
Record 38
Richard Nixon
http://en.wikipedia.org/wiki/Richard_Nixon
---
Record 39
Gerald Ford
http://en.wikipedia.org/wiki/Gerald_Ford
---
Record 40
Jimmy Carter
http://en.wikipedia.org/wiki/Jimmy_Carter

and I would like to print it out like

Record 38 \t Richard Nixon \t http://en.wikipedia.org/wiki/Richard_Nixon
Record 39 \t Gerald Ford \t http://en.wikipedia.org/wiki/Gerald_Ford
Record 40 \t Jimmy Carter \t http://en.wikipedia.org/wiki/Jimmy_Carter



$ echo "---
Record 38
Richard Nixon
http://en.wikipedia.org/wiki/Richard_Nixon
---
Record 39
Gerald Ford
http://en.wikipedia.org/wiki/Gerald_Ford
---
Record 40
Jimmy Carter
http://en.wikipedia.org/wiki/Jimmy_Carter
" | perl -e'

$/ = ( "-" x 27 ) . "\n";
$\ = "\n";

while ( <> ) {
chomp;
tr/\n/\t/;
print;
}
'

Record 38   Richard Nixon   http://en.wikipedia.org/wiki/Richard_Nixon
Record 39   Gerald Ford http://en.wikipedia.org/wiki/Gerald_Ford
Record 40   Jimmy Carterhttp://en.wikipedia.org/wiki/Jimmy_Carter



John
--
Any intelligent fool can make things bigger and
more complex... It takes a touch of genius -
and a lot of courage to move in the opposite
direction.   -- Albert Einstein

--
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/

Re: Appending extra characters to file name

[Igor Dovgiy]

Hi Alex,

Your script should be like that:

$oldfile = $file;
chomp($oldfile);
my $newfile = $oldfile . ".x";

... as $file (and $oldfile, which is redundant btw) ends with "\n".

Besides, it'd be much easier to collect all the files in the given
directory with fileglob.

-- iD


2012/1/5 Alex Ardin 

> Hi,
> I frequently have to review logs that are captured with 'script' using
> terminal but these logs have many control characters.
>
> The intent of the script is the following:
>
> 1. Get a listing of all the logs in a directory, storing the list of
> files in an array.
> 2. For each filename in the array, create a new filename but append a
> ".x" to the original filename, so that file 'a' would result in a new
> file 'a.x', as an example.
> 3. Open the original file, search and remove any control characters
> (such as  ^M, ^C, and so on) line by line and store the results in the
> newly created file ending in .x (such as the a.x example above).
> 4. After reaching the end file, close both files.
> 5. Move the newly created .x file so that it replaces the original
> file (or 'mv a.x a' in the example above).
>
> What happens is that the script works fine, all control characters are
> replaced in the file, but the move fails, so that if I start with a
> directory containing these files:
>
> xong$ ls -rlt
> -rw-r--r--  1 rootroot  staff  50033 Oct 11 17:43 xyz
> -rw-r--r--  1 rootroot  staff  50033 Oct 13 18:57 123
>
> when the script finishes executing, I end up with these file, the
> first two are the original files with all the control characters, the
> last two are the newly created files with the control characters
> removed:
>
> xong$ ls -rlt
> -rw-r--r--  1 rootroot  staff  50033 Oct 11 17:43 xyz
> -rw-r--r--  1 rootroot  staff  50033 Oct 13 18:57 123
> -rw-r--r--  1 rootroot  staff  49792 Jan  4 22:14 xyz?.x
> -rw-r--r--  1 rootroot  staff  49792 Jan  4 22:14 123?.x
>
> Here's the errors I see when the script executes, the comments are
> just for debugging:
>
> Here's a listing of the files:
> 123
> xyz
>
> Here's newfile: 123
> .xHere's oldfile: 123
>
> Now mv file
> Here's newfile: 123
> .xHere's oldfile: 123
> usage: mv [-f | -i | -n] [-v] source target
>   mv [-f | -i | -n] [-v] source ... directory
> sh: line 1: .x: command not found
>
> I'm not sure why this is happening, the newfile should contain "123.x"
> but instead the .x is appearing on the newline.  I've gone through the
> "Programming Perl" book and can't seem to fine another way to do this,
> and I've also searched this group, and didn't find anything similar.
>
> Here's the script:
>
> #!/usr/bin/perl
>
> #
> # Delete all control characters in a group of files in a directory.
> #
>
> #use strict;
> use warnings;
>
> if ($#ARGV != 0) {
>print "\nusage: get_file_list.pl directory\n\n";
>exit;
> }
>
> $dir_path = $ARGV[0];
> chdir "$dir_path" or die "\ncd failed, check file path!\n\n";
>
> $cmd = "pwd";
> if(system($cmd)) { print "\n\npwd failed.\n"; }
>
> $cmd = "ls -l";
> if(system($cmd)) { print "\n\nFile listing failed.\n"; }
>
> # Store file listing in a array:
> $cmd = "ls";
> my @fl = qx($cmd);
>
> print "\nHere's a listing of the files:\n";
>
> foreach $file (@fl) {
>print $file;
> }
>
> I thought it was a result of my foreach loop but the loop exits after
> the last file is reached, so there's nothing to force a newline when
> my script copies the original filename to a new file name ending in
> ".x".
>
> foreach $file (@fl) {
>$oldfile = $file;
>my $newfile = $oldfile . ".x";
>print("Here's newfile: $newfile");
>print("Here's oldfile: $oldfile");
>
>open(OF, $oldfile);
>open(NF, ">$newfile");
>
># read in each line of the file
>while ($line = ) {
>$line =~ s/\cM//g;
>$line =~ s/\cH//g;
>$line =~ s/\cC//g;
>$line =~ s/\cL//g;
>$line =~ s/\cG//g;
>$line =~ s/\cP//g;
>$line =~ s/\cQ//g;
>$line =~ s/\cR//g;
>$line =~ s/\c[//g;
>print NF $line;
>}
>
>print("\nNow mv file\n");
>print("Here's newfile2: $newfile");
>print("Here's oldfile2: $oldfile");
>
>$cmd = "mv $newfile $oldfile";
>if(system($cmd)) { print "rename failed\n"; }
>
>close(OF);
>close(NF);
> }
>
> I also thought it had been the result of my closing my files after I
> attempted the move, but I get the same results if I move these lines:
>
>close(OF);
>close(NF);
>
> above this line:
>
> print("\nNow mv file\n");
>
> so that the files are closed and then attempt to move them.
>
> Is it possible to understand how to work around this?
>
> Alex
>
>
> --
> To unsubscribe, e-mail: beginners-unsubscr...@perl.org
> For additional commands, e-mail: beginners-h...@perl.org
> http://learn.perl.org/
>
>
>

Appending extra characters to file name

[Alex Ardin]

Hi,
I frequently have to review logs that are captured with 'script' using
terminal but these logs have many control characters.

The intent of the script is the following:

1. Get a listing of all the logs in a directory, storing the list of
files in an array.
2. For each filename in the array, create a new filename but append a
".x" to the original filename, so that file 'a' would result in a new
file 'a.x', as an example.
3. Open the original file, search and remove any control characters
(such as  ^M, ^C, and so on) line by line and store the results in the
newly created file ending in .x (such as the a.x example above).
4. After reaching the end file, close both files.
5. Move the newly created .x file so that it replaces the original
file (or 'mv a.x a' in the example above).

What happens is that the script works fine, all control characters are
replaced in the file, but the move fails, so that if I start with a
directory containing these files:

xong$ ls -rlt
-rw-r--r--  1 rootroot  staff  50033 Oct 11 17:43 xyz
-rw-r--r--  1 rootroot  staff  50033 Oct 13 18:57 123

when the script finishes executing, I end up with these file, the
first two are the original files with all the control characters, the
last two are the newly created files with the control characters
removed:

xong$ ls -rlt
-rw-r--r--  1 rootroot  staff  50033 Oct 11 17:43 xyz
-rw-r--r--  1 rootroot  staff  50033 Oct 13 18:57 123
-rw-r--r--  1 rootroot  staff  49792 Jan  4 22:14 xyz?.x
-rw-r--r--  1 rootroot  staff  49792 Jan  4 22:14 123?.x

Here's the errors I see when the script executes, the comments are
just for debugging:

Here's a listing of the files:
123
xyz

Here's newfile: 123
.xHere's oldfile: 123

Now mv file
Here's newfile: 123
.xHere's oldfile: 123
usage: mv [-f | -i | -n] [-v] source target
   mv [-f | -i | -n] [-v] source ... directory
sh: line 1: .x: command not found

I'm not sure why this is happening, the newfile should contain "123.x"
but instead the .x is appearing on the newline.  I've gone through the
"Programming Perl" book and can't seem to fine another way to do this,
and I've also searched this group, and didn't find anything similar.

Here's the script:

#!/usr/bin/perl

#
# Delete all control characters in a group of files in a directory.
#

#use strict;
use warnings;

if ($#ARGV != 0) {
print "\nusage: get_file_list.pl directory\n\n";
exit;
}

$dir_path = $ARGV[0];
chdir "$dir_path" or die "\ncd failed, check file path!\n\n";

$cmd = "pwd";
if(system($cmd)) { print "\n\npwd failed.\n"; }

$cmd = "ls -l";
if(system($cmd)) { print "\n\nFile listing failed.\n"; }

# Store file listing in a array:
$cmd = "ls";
my @fl = qx($cmd);

print "\nHere's a listing of the files:\n";

foreach $file (@fl) {
print $file;
}

I thought it was a result of my foreach loop but the loop exits after
the last file is reached, so there's nothing to force a newline when
my script copies the original filename to a new file name ending in
".x".

foreach $file (@fl) {
$oldfile = $file;
my $newfile = $oldfile . ".x";
print("Here's newfile: $newfile");
print("Here's oldfile: $oldfile");

open(OF, $oldfile);
open(NF, ">$newfile");

# read in each line of the file
while ($line = ) {
$line =~ s/\cM//g;
$line =~ s/\cH//g;
$line =~ s/\cC//g;
$line =~ s/\cL//g;
$line =~ s/\cG//g;
$line =~ s/\cP//g;
$line =~ s/\cQ//g;
$line =~ s/\cR//g;
$line =~ s/\c[//g;
print NF $line;
}

print("\nNow mv file\n");
print("Here's newfile2: $newfile");
print("Here's oldfile2: $oldfile");

$cmd = "mv $newfile $oldfile";
if(system($cmd)) { print "rename failed\n"; }

close(OF);
close(NF);
}

I also thought it had been the result of my closing my files after I
attempted the move, but I get the same results if I move these lines:

close(OF);
close(NF);

above this line:

print("\nNow mv file\n");

so that the files are closed and then attempt to move them.

Is it possible to understand how to work around this?

Alex


-- 
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/

Splitting and printing on a single line

[bob]

Hello

I am very new to perl and I need some guidance on the following issue

My data set is like this

---
Record 38
Richard Nixon
http://en.wikipedia.org/wiki/Richard_Nixon
---
Record 39
Gerald Ford
http://en.wikipedia.org/wiki/Gerald_Ford
---
Record 40
Jimmy Carter
http://en.wikipedia.org/wiki/Jimmy_Carter

and I would like to print it out like

Record 38 \t Richard Nixon \t http://en.wikipedia.org/wiki/Richard_Nixon
Record 39 \t Gerald Ford \t http://en.wikipedia.org/wiki/Gerald_Ford
Record 40 \t Jimmy Carter \t http://en.wikipedia.org/wiki/Jimmy_Carter

Thank you


-- 
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/