RE: removing duplicate lines across files.

[Siva Prasad]

Hi Gurus,

 

Sorry I did not give detailed description of the problem, Here it goes

 

 

I have a array of file names (file1, file2, file3, file4).

 

Each  the duplicates should be removed in the below fashion

 

The lines in file1 should be removed from file2,fiile3,file4,If  they
exists.

The lines in file2 should be removed from file3,file4 if They exists,

The lines in file3 should be removed from file4 if they exists.

 

Is there any way apart from forloop

 

Thanks in Advance

Siva

 

 

 

 

-Original Message-
From: John W. Krahn [mailto:[EMAIL PROTECTED] 
Sent: Thursday, January 03, 2008 4:04 PM
To: Perl Beginners
Subject: Re: removing duplicate lines across files.

 

Siva Prasad wrote:

> 

> Hi Gurus,

 

Hello,

 

> I want to remove duplicate lines across files.

> 

> Below is the detailed problem,

> 

> 

> I have file1 file2,file3,file4,file5,

> 

> The lines in file1 are there in file2,file3,file4,file5  I want to remove

> all the lines which are there in file1 from file2,file3,file4,file5.

> 

> Can anybody please tell me to solve the above problem?

 

UNTESTED:

 

#!/usr/bin/perl

use warnings;

use strict;

 

my $file = 'file1';

 

open my $fh, '<', $file or die "Cannot open '$file' $!";

 

my %lines;

@lines{ <$fh> } = ();

 

close $fh;

 

{   local ( $^I, @ARGV ) = qw/ .bak file2 file3 file4 file5 /;

 

 while ( <> ) {

 next if exists $lines{ $_ };

 print;

 }

 }

 

__END__

 

 

 

John

-- 

Perl isn't a toolbox, but a small machine shop where you

can special-order certain sorts of tools at low cost and

in short order.-- Larry Wall

 

-- 

To unsubscribe, e-mail: [EMAIL PROTECTED]

For additional commands, e-mail: [EMAIL PROTECTED]

http://learn.perl.org/

 

Re: removing duplicate lines across files.

[Rob Dixon]

Siva Prasad wrote:


I want to remove duplicate lines across files.

Below is the detailed problem,

 


I have file1 file2,file3,file4,file5,

The lines in file1 are there in file2,file3,file4,file5  I want to 
remove all the lines which are there in file1 from file2,file3,file4,file5.


Can anybody please tell me to solve the above problem?


You have been asked already not to makes posts in blue lettering. Please
have the courtesy to post in a fashion appropriate for the list.

read the faq article

  perldoc -q duplicate elements

which describes how to do this with an array of data. Hopefully you can
make the step to doing the same thing with disk files.

HTH,

Rob

--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/

Re: removing duplicate lines across files.

[John W. Krahn]

Siva Prasad wrote:


Hi Gurus,


Hello,


I want to remove duplicate lines across files.

Below is the detailed problem,


I have file1 file2,file3,file4,file5,

The lines in file1 are there in file2,file3,file4,file5  I want to remove
all the lines which are there in file1 from file2,file3,file4,file5.

Can anybody please tell me to solve the above problem?


UNTESTED:

#!/usr/bin/perl
use warnings;
use strict;

my $file = 'file1';

open my $fh, '<', $file or die "Cannot open '$file' $!";

my %lines;
@lines{ <$fh> } = ();

close $fh;

{   local ( $^I, @ARGV ) = qw/ .bak file2 file3 file4 file5 /;

while ( <> ) {
next if exists $lines{ $_ };
print;
}
}

__END__



John
--
Perl isn't a toolbox, but a small machine shop where you
can special-order certain sorts of tools at low cost and
in short order.-- Larry Wall

--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/

removing duplicate lines across files.

[Siva Prasad]

 

Hi Gurus,

 

I want to remove duplicate lines across files.

 

Below is the detailed problem,

 

 

I have file1 file2,file3,file4,file5,

 

The lines in file1 are there in file2,file3,file4,file5  I want to remove
all the lines which are there in file1 from file2,file3,file4,file5.

 

Can anybody please tell me to solve the above problem?

 

 

 

 

Thanks in Advance,

Siva

<>

Re: removing duplicate lines

[R. Joseph Newton]

Andrew Gaffney wrote:

> John W. Krahn wrote:
> > Whenever you want unique values think "hash".
>
> Well, it would have been weird to have a hash with keys named 'NET USE F:
> SKYLINE\\SKYLINEF\r\n'.

No.  It is not at all wierd to use hash for any of the puroses for which it is 
well-suited.
Among of those purposes are ensuring unique .ness and testing existence.  1-valued 
hashes are
extremely efficient engines fr both

my $paths_seen = {};
foreach $drive (keys drives) {
   my $map_command = $drive{$_};
   next if $paths_seen->{$map_command);
   assign_drive($map_command);
   $paths_seen->{$map_command) = 1;
}

The above assumes that you are willing to make brute-force assumptions about which 
mapping to a
particular share is the appropriate one.  If there are factors you wish to weigh in 
chosing which
mapping to keep, you would want to call some resolving function when an element is 
found rather
than simply next-ing.

Joseph



-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: removing duplicate lines

[James Edward Gray II]

On Dec 9, 2003, at 8:33 PM, Andrew Gaffney wrote:

John W. Krahn wrote:
Andrew Gaffney wrote:
I am writing a Perl script to automatically generate a netlogon.bat 
file for Samba
whenever a user logs onto a domain. The only parameter that is 
passes to it is the
username. My problem is that different groups get some of the same 
mappings. What I really
need to do is filter out duplicate lines in the finished output.
Whenever you want unique values think "hash".
Well, it would have been weird to have a hash with keys named 'NET USE 
F: SKYLINE\\SKYLINEF\r\n'.
Why is this weird?  It's a string, hash keys need to be strings.  Good 
fit.

Actually, this is a very common textbook Perl idiom.  The sooner you 
get the hang of tricks like this the better.

James

--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: removing duplicate lines

[Andrew Gaffney]

John W. Krahn wrote:
Andrew Gaffney wrote:

I am writing a Perl script to automatically generate a netlogon.bat file for Samba
whenever a user logs onto a domain. The only parameter that is passes to it is the
username. My problem is that different groups get some of the same mappings. What I 
really
need to do is filter out duplicate lines in the finished output.
Whenever you want unique values think "hash".
Well, it would have been weird to have a hash with keys named 'NET USE F: 
SKYLINE\\SKYLINEF\r\n'.

I tried piping the output
through 'uniq' but it only filters successive duplicate lines. Anyone have any 
suggestions?
Your example does not have 'uniq' (or 'sort -u') in it so I am not sure
what you are trying to do.
I mean that I tried 'cat /tmp/user.bat > uniq' after the script had run to see how it 
would work.

#!/usr/bin/perl
use warnings;
use strict;
my $user = shift;
my $drives = {F => "NET USE F: SKYLINE\\SKYLINEF\r\n",
  H => "NET USE H: SKYLINE\\SHARE\r\n",
  I => "NET USE I: SHIPPING1\\INVENTORY\r\n",
  M => "NET USE M: SKYLINE\\SKYLINEM\r\n",
  S => "NET USE S: SHIPPING1\\SHOP\r\n",
  Y => "NET USE Y: ACCOUNTING\\FLTSCHOOL\r\n",
  Z => "NET USE Z: ACCOUNTING\\MAINT\r\n"};
Why not just use a hash instead of a reference to a hash?  The use of
"\r\n" is non-portable, you should use "\015\012" instead.
I'm not that worried about portability since this was something I threw together for use 
on MY system with MY particular setup.

my $CRLF = "\015\012";

my %drives = (
F => 'NET USE F: \\SKYLINE\SKYLINEF' . $CRLF,
H => 'NET USE H: \\SKYLINE\SHARE' . $CRLF,
I => 'NET USE I: \\SHIPPING1\INVENTORY' . $CRLF,
M => 'NET USE M: \\SKYLINE\SKYLINEM' . $CRLF,
S => 'NET USE S: \\SHIPPING1\SHOP' . $CRLF,
Y => 'NET USE Y: \\ACCOUNTING\FLTSCHOOL' . $CRLF,
Z => 'NET USE Z: \\ACCOUNTING\MAINT' . $CRLF,
);
my $which = {accounting => "F H I M S Y Z", mech => "I M S Z", dispatch => "M",
instructors => "M"};
You should probably use a hash of arrays for this (so you don't have to
split the string later):
my %which = (
accounting  => [ qw(F H I M S Y Z) ],
mech=> [ qw(I M S Z) ],
dispatch=> [ qw(M) ],
instructors => [ qw(M) ],
);
I'll probably change this.

my $groups = `cat /etc/group | grep ${user} | cut -d ':' -f 1`;
Ick, ick, ick!  Perl provides built-in functions to access /etc/group
and /etc/passwd
perldoc -f getgrnam
perldoc -f getgrgid
perldoc -f getgrent
perldoc -f setgrent
perldoc -f endgrent
perldoc -f getpwnam
perldoc -f getpwuid
perldoc -f getpwent
perldoc -f setpwent
perldoc -f endpwent
This script wasn't much more than a quick hack, anyway. I'll work on stuff like that later.

$groups =~ s/\n/\:/sg;

# Start generating logon script
#open LOGON, ">/usr/local/samba/netlogon/${user}.bat";
open LOGON, ">/tmp/${user}.bat";
You should _ALWAYS_ verify that the file opened correctly.

open LOGON, ">/tmp/$user.bat" or die "Cannot open /tmp/$user.bat: $!";
I agree. I just modified existing code and didn't think about that.

print LOGON "[EMAIL PROTECTED] OFF\r\n";

foreach $group (split /:/, $groups) {
  foreach $drive (split / /, $which->{$group}) {
print LOGON $drives->{$drive};
  }
}
close LOGON;
John
Thanks for all the suggestions.

--
Andrew Gaffney
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: removing duplicate lines

[John W. Krahn]

Andrew Gaffney wrote:
> 
> I am writing a Perl script to automatically generate a netlogon.bat file for Samba
> whenever a user logs onto a domain. The only parameter that is passes to it is the
> username. My problem is that different groups get some of the same mappings. What I 
> really
> need to do is filter out duplicate lines in the finished output.

Whenever you want unique values think "hash".


> I tried piping the output
> through 'uniq' but it only filters successive duplicate lines. Anyone have any 
> suggestions?

Your example does not have 'uniq' (or 'sort -u') in it so I am not sure
what you are trying to do.


> #!/usr/bin/perl

use warnings;
use strict;

> my $user = shift;
> my $drives = {F => "NET USE F: SKYLINE\\SKYLINEF\r\n",
>H => "NET USE H: SKYLINE\\SHARE\r\n",
>I => "NET USE I: SHIPPING1\\INVENTORY\r\n",
>M => "NET USE M: SKYLINE\\SKYLINEM\r\n",
>S => "NET USE S: SHIPPING1\\SHOP\r\n",
>Y => "NET USE Y: ACCOUNTING\\FLTSCHOOL\r\n",
>Z => "NET USE Z: ACCOUNTING\\MAINT\r\n"};

Why not just use a hash instead of a reference to a hash?  The use of
"\r\n" is non-portable, you should use "\015\012" instead.

my $CRLF = "\015\012";

my %drives = (
F => 'NET USE F: \\SKYLINE\SKYLINEF' . $CRLF,
H => 'NET USE H: \\SKYLINE\SHARE' . $CRLF,
I => 'NET USE I: \\SHIPPING1\INVENTORY' . $CRLF,
M => 'NET USE M: \\SKYLINE\SKYLINEM' . $CRLF,
S => 'NET USE S: \\SHIPPING1\SHOP' . $CRLF,
Y => 'NET USE Y: \\ACCOUNTING\FLTSCHOOL' . $CRLF,
Z => 'NET USE Z: \\ACCOUNTING\MAINT' . $CRLF,
);


> my $which = {accounting => "F H I M S Y Z", mech => "I M S Z", dispatch => "M",
> instructors => "M"};

You should probably use a hash of arrays for this (so you don't have to
split the string later):

my %which = (
accounting  => [ qw(F H I M S Y Z) ],
mech=> [ qw(I M S Z) ],
dispatch=> [ qw(M) ],
instructors => [ qw(M) ],
);


> my $groups = `cat /etc/group | grep ${user} | cut -d ':' -f 1`;

Ick, ick, ick!  Perl provides built-in functions to access /etc/group
and /etc/passwd

perldoc -f getgrnam
perldoc -f getgrgid
perldoc -f getgrent
perldoc -f setgrent
perldoc -f endgrent

perldoc -f getpwnam
perldoc -f getpwuid
perldoc -f getpwent
perldoc -f setpwent
perldoc -f endpwent


> $groups =~ s/\n/\:/sg;
> 
> # Start generating logon script
> #open LOGON, ">/usr/local/samba/netlogon/${user}.bat";
> open LOGON, ">/tmp/${user}.bat";

You should _ALWAYS_ verify that the file opened correctly.

open LOGON, ">/tmp/$user.bat" or die "Cannot open /tmp/$user.bat: $!";


> print LOGON "[EMAIL PROTECTED] OFF\r\n";
> 
> foreach $group (split /:/, $groups) {
>foreach $drive (split / /, $which->{$group}) {
>  print LOGON $drives->{$drive};
>}
> }
> 
> close LOGON;



John
-- 
use Perl;
program
fulfillment

-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: removing duplicate lines

[Rob Dixon]

Andrew Gaffney wrote:
>
> Rob Dixon wrote:
> >
> > Andrew Gaffney wrote:
> >
> > > I am writing a Perl script to automatically generate a netlogon.bat file for 
> > > Samba
> > > whenever a user logs onto a domain. The only parameter that is passes to it is 
> > > the
> > > username. My problem is that different groups get some of the same mappings. 
> > > What I really
> > > need to do is filter out duplicate lines in the finished output. I tried piping 
> > > the output
> > > through 'uniq' but it only filters successive duplicate lines. Anyone have any 
> > > suggestions?
> >
> >
> > [snip code]
> >
> > Hi Andrew.
> >
> > The quick answer is:
> >
> >   perldoc -q dupl
> >
> > If you need any more then ask again :)
>
> I was able to indirectly get the answer from that. Reading that,
> I realized that if I run my output through 'sort' and then 'uniq'
> or even just 'sort -u', it does what I want it to do.

I'm reluctant to let this go, but a 'proper' answer would have to be
of the, "I wouldn't start from here, " type. You've used Perl as
a scripting language, which it isn't. Perl's very good at doing anything
you need on a platform-independent basis, and shelling out with 'system'
calls or backticks is almost never necessary and makes the whole program
platform and shell-specific.

If this is even a semi-permanent piece of software then, if I were
you, I would let the group blitz it just to show you what can be done.
You might even want to see that anyway as a learning exercise.

HTH,

Rob



-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: removing duplicate lines

[Andrew Gaffney]

Rob Dixon wrote:
Andrew Gaffney wrote:

I am writing a Perl script to automatically generate a netlogon.bat file for Samba
whenever a user logs onto a domain. The only parameter that is passes to it is the
username. My problem is that different groups get some of the same mappings. What I 
really
need to do is filter out duplicate lines in the finished output. I tried piping the 
output
through 'uniq' but it only filters successive duplicate lines. Anyone have any 
suggestions?


[snip code]

Hi Andrew.

The quick answer is:

  perldoc -q dupl

If you need any more then ask again :)
I was able to indirectly get the answer from that. Reading that, I realized that if I run 
my output through 'sort' and then 'uniq' or even just 'sort -u', it does what I want it to do.

--
Andrew Gaffney
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: removing duplicate lines

[Rob Dixon]

Andrew Gaffney wrote:
>
> I am writing a Perl script to automatically generate a netlogon.bat file for Samba
> whenever a user logs onto a domain. The only parameter that is passes to it is the
> username. My problem is that different groups get some of the same mappings. What I 
> really
> need to do is filter out duplicate lines in the finished output. I tried piping the 
> output
> through 'uniq' but it only filters successive duplicate lines. Anyone have any 
> suggestions?

[snip code]

Hi Andrew.

The quick answer is:

  perldoc -q dupl

If you need any more then ask again :)

Rob



-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

removing duplicate lines

[Andrew Gaffney]

I am writing a Perl script to automatically generate a netlogon.bat file for Samba 
whenever a user logs onto a domain. The only parameter that is passes to it is the 
username. My problem is that different groups get some of the same mappings. What I really 
need to do is filter out duplicate lines in the finished output. I tried piping the output 
through 'uniq' but it only filters successive duplicate lines. Anyone have any suggestions?

#!/usr/bin/perl

my $user = shift;
my $drives = {F => "NET USE F: SKYLINE\\SKYLINEF\r\n",
  H => "NET USE H: SKYLINE\\SHARE\r\n",
  I => "NET USE I: SHIPPING1\\INVENTORY\r\n",
  M => "NET USE M: SKYLINE\\SKYLINEM\r\n",
  S => "NET USE S: SHIPPING1\\SHOP\r\n",
  Y => "NET USE Y: ACCOUNTING\\FLTSCHOOL\r\n",
  Z => "NET USE Z: ACCOUNTING\\MAINT\r\n"};
my $which = {accounting => "F H I M S Y Z", mech => "I M S Z", dispatch => "M", 
instructors => "M"};
my $groups = `cat /etc/group | grep ${user} | cut -d ':' -f 1`;
$groups =~ s/\n/\:/sg;

# Start generating logon script
#open LOGON, ">/usr/local/samba/netlogon/${user}.bat";
open LOGON, ">/tmp/${user}.bat";
print LOGON "[EMAIL PROTECTED] OFF\r\n";
foreach $group (split /:/, $groups) {
  foreach $drive (split / /, $which->{$group}) {
print LOGON $drives->{$drive};
  }
}
close LOGON;

--
Andrew Gaffney
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: Removing duplicate lines.

[John W. Krahn]

Hi Jenda  :-)

Jenda Krynicky wrote:
> 
> Is it safe to assume that all duplicates are together like this? If
> so all you have to do is to
> 1) read the file line by line
> 2) only print the line you just read if it's different from the last
> one
> 3) remember the line
> 
> or if I word it differently.
> 1) read the file
> 2) skip the line if it's the same as the last one
> 3) print it and remember it
> 
> my $last = '';
> while (<>) {
> next if $_ eq $last;
> print $_;
> $last = $;
  ^^
Assigning the value of $; to $last will not do what you want.  Note that
it is not a syntax error as $; is a valid variable name and the
semicolon is not required at the end of the block.

  $last = $_


> }


John
-- 
use Perl;
program
fulfillment

-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

RE: Removing duplicate lines.

[jonathan . musto]

Thanks all, that worked a treat.

Jonathan Musto

 

BT Global Services
Telephone - 0113 237 3277
Fax - 0113 244 1413
E-mail - [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> 
http://www.technet.bt.com/sit/public


British Telecommunications plc 
Registered office: 81 Newgate Street London EC1A 7AJ 
Registered in England no. 180 
This electronic message contains information from British Telecommunications
plc which may be privileged or  confidential. The information is intended to
be for the use of the individual(s) or entity named above. If you  are not
the intended recipient be aware that any disclosure, copying, distribution
or use of the contents of  this information is prohibited. If you have
received this electronic message in error, please notify us by  telephone or
email (to the numbers or address above) immediately.






-Original Message-
From: Jenda Krynicky [mailto:[EMAIL PROTECTED]
Sent: Monday, July 21, 2003 13:33
To: [EMAIL PROTECTED]
Subject: Re: Removing duplicate lines.


From: [EMAIL PROTECTED]
> I have a text file which contains a list of companies:
> 
> NORTH DOWN AND ARDS INSTITUTE
> NOTTINGHAM HEALTH AUTHORITY
> 1ST CONTACT GROUP LTD
> 1ST CONTACT GROUP LTD
> 1ST CONTACT GROUP LTD
> 1ST CONTACT GROUP LTD
> 4D TELECOM & KINGSTON INMEDIA
> A E COOK LTD
> A E COOK LTD
> 
> etc..
> 
> How can a write a simple perl script to remove the duplicates and
> leave just one of each customer? Any help would be great.

Is it safe to assume that all duplicates are together like this? If 
so all you have to do is to
1) read the file line by line
2) only print the line you just read if it's different from the last

one
3) remember the line

or if I word it differently. 
1) read the file
2) skip the line if it's the same as the last one
3) print it and remember it

my $last = '';
while (<>) {
next if $_ eq $last;
print $_;
$last = $;
}

If the duplicates are scattered all over the place, then the easiest 
solution is to use a hash, That will get rid of the duplicates for 
you:

while (<>) {
chomp;
$seen{$_}++;
}

foreack my $item (keys %seen) {
print $item,"\n";
}

If the list of companies is huge you may need to store the hash on 
disk to prevent swapping:

use DB_File;
tie %seen, 'DB_File', $filename;
...

Jenda
= [EMAIL PROTECTED] === http://Jenda.Krynicky.cz =
When it comes to wine, women and song, wizards are allowed 
to get drunk and croon as much as they like.
-- Terry Pratchett in Sourcery


-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Removing duplicate lines.

[Jenda Krynicky]

From: [EMAIL PROTECTED]
> I have a text file which contains a list of companies:
> 
> NORTH DOWN AND ARDS INSTITUTE
> NOTTINGHAM HEALTH AUTHORITY
> 1ST CONTACT GROUP LTD
> 1ST CONTACT GROUP LTD
> 1ST CONTACT GROUP LTD
> 1ST CONTACT GROUP LTD
> 4D TELECOM & KINGSTON INMEDIA
> A E COOK LTD
> A E COOK LTD
> 
> etc..
> 
> How can a write a simple perl script to remove the duplicates and
> leave just one of each customer? Any help would be great.

Is it safe to assume that all duplicates are together like this? If 
so all you have to do is to
1) read the file line by line
2) only print the line you just read if it's different from the last 
one
3) remember the line

or if I word it differently. 
1) read the file
2) skip the line if it's the same as the last one
3) print it and remember it

my $last = '';
while (<>) {
next if $_ eq $last;
print $_;
$last = $;
}

If the duplicates are scattered all over the place, then the easiest 
solution is to use a hash, That will get rid of the duplicates for 
you:

while (<>) {
chomp;
$seen{$_}++;
}

foreack my $item (keys %seen) {
print $item,"\n";
}

If the list of companies is huge you may need to store the hash on 
disk to prevent swapping:

use DB_File;
tie %seen, 'DB_File', $filename;
...

Jenda
= [EMAIL PROTECTED] === http://Jenda.Krynicky.cz =
When it comes to wine, women and song, wizards are allowed 
to get drunk and croon as much as they like.
-- Terry Pratchett in Sourcery


-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Removing duplicate lines.

[Janek Schleicher]

Jonathan Musto wrote at Mon, 21 Jul 2003 13:11:10 +0100:

> I have a text file which contains a list of companies:
>  
> NORTH DOWN AND ARDS INSTITUTE
> NOTTINGHAM HEALTH AUTHORITY
> 1ST CONTACT GROUP LTD
> 1ST CONTACT GROUP LTD
> 1ST CONTACT GROUP LTD
> 1ST CONTACT GROUP LTD
> 4D TELECOM & KINGSTON INMEDIA
> A E COOK LTD
> A E COOK LTD
>  
> etc..
>  
> How can a write a simple perl script to remove the duplicates and leave just
> one of each customer?
> Any help would be great.

What have you tried so far?
Have you read
perldoc -q duplicate

[untested]
my %seen;
while (<>) {
   print unless $seen{$_}++;
}


Greetings,
Janek

-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Removing duplicate lines.

[jonathan . musto]

I have a text file which contains a list of companies:
 
NORTH DOWN AND ARDS INSTITUTE
NOTTINGHAM HEALTH AUTHORITY
1ST CONTACT GROUP LTD
1ST CONTACT GROUP LTD
1ST CONTACT GROUP LTD
1ST CONTACT GROUP LTD
4D TELECOM & KINGSTON INMEDIA
A E COOK LTD
A E COOK LTD
 
etc..
 
How can a write a simple perl script to remove the duplicates and leave just
one of each customer?
Any help would be great.
 
Regards,
 
 
Jonathan Musto

 

BT Global Services
Telephone - 0113 237 3277
Fax - 0113 244 1413
E-mail -   [EMAIL PROTECTED]
http://www.technet.bt.com/sit/public  


British Telecommunications plc 
Registered office: 81 Newgate Street London EC1A 7AJ 
Registered in England no. 180 
This electronic message contains information from British Telecommunications
plc which may be privileged or  confidential. The information is intended to
be for the use of the individual(s) or entity named above. If you  are not
the intended recipient be aware that any disclosure, copying, distribution
or use of the contents of  this information is prohibited. If you have
received this electronic message in error, please notify us by  telephone or
email (to the numbers or address above) immediately.