Re: remove duplicate lines

2007-01-11 Thread oryann9
Dr.Ruud [EMAIL PROTECTED] wrote: beast schreef: a 100 a 102 c 100 a 102 b 111 c 100 c 102 c 100 c 100 a 102 ... I would like to have a list (either array or hash) with unique line . perl -ne'$_{$_}||=print' datafile or perl -pe'$_ x=!$$_++' datafile -- Affijn, Ruud what

Re: remove duplicate lines

2007-01-11 Thread Dr.Ruud
oryann9 schreef: Dr.Ruud [EMAIL PROTECTED] wrote: beast schreef: beast: a 100 a 102 c 100 a 102 b 111 c 100 c 102 c 100 c 100 a 102 ... I would like to have a list (either array or hash) with unique line . perl -ne'$_{$_}||=print' datafile what are these exactly doing in

Re: remove duplicate lines

2007-01-11 Thread Jason Roth
If we're just going for confusing concise one liners then I would use perl -ne '$$_||=print' You save three whole characters by using the symbol table instead of a hash :) On 1/11/07, Dr.Ruud [EMAIL PROTECTED] wrote: oryann9 schreef: Dr.Ruud [EMAIL PROTECTED] wrote: beast schreef: beast:

Re: remove duplicate lines

2007-01-11 Thread oryann9
Dr.Ruud [EMAIL PROTECTED] wrote:oryann9 schreef: Dr.Ruud wrote: beast schreef: beast: a 100 a 102 c 100 a 102 b 111 c 100 c 102 c 100 c 100 a 102 ... I would like to have a list (either array or hash) with unique line . perl -ne'$_{$_}||=print' datafile what are these

Re: remove duplicate lines

2007-01-11 Thread Dr.Ruud
Dr.Ruud schreef: beast: a 100 a 102 ... c 100 a 102 ... I would like to have a list (either array or hash) with unique line . perl -ne'$_{$_}||=print' datafile Jason Roth's version: perl -ne'$$_||=print' datafile or perl -pe'$_ x=!$$_++' datafile -- Affijn, Ruud Gewoon

Re: remove duplicate lines

2007-01-11 Thread beast
Dr.Ruud wrote: while ( ARGV ) { next if $d{$_}; print; $d{$_} = 1; } Got the idea. Many thanks! -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] http://learn.perl.org/

Re: remove duplicate lines

2007-01-10 Thread Mumia W.
On 01/10/2007 01:35 AM, beast wrote: [...] It will only remove duplicate key. Is this still acceptable in perl (its very ugly =( [...] This would remove duplicate lines: use List::MoreUtils qw(uniq); use File::Slurp; my @list = uniq read_file 'rm_dup_lines.txt'; print

Re: remove duplicate lines

2007-01-10 Thread beast
Mumia W. wrote: On 01/10/2007 01:35 AM, beast wrote: [...] It will only remove duplicate key. Is this still acceptable in perl (its very ugly =( [...] This would remove duplicate lines: use List::MoreUtils qw(uniq); use File::Slurp; my @list = uniq read_file 'rm_dup_lines.txt

Re: remove duplicate lines

2007-01-10 Thread Mathew
beast wrote: Mumia W. wrote: On 01/10/2007 01:35 AM, beast wrote: [...] It will only remove duplicate key. Is this still acceptable in perl (its very ugly =( [...] This would remove duplicate lines: use List::MoreUtils qw(uniq); use File::Slurp; my @list = uniq read_file

Re: remove duplicate lines

2007-01-10 Thread Dr.Ruud
beast schreef: a 100 a 102 c 100 a 102 b 111 c 100 c 102 c 100 c 100 a 102 ... I would like to have a list (either array or hash) with unique line . perl -ne'$_{$_}||=print' datafile or perl -pe'$_ x=!$$_++' datafile -- Affijn, Ruud Gewoon is een tijger. -- To

remove duplicate lines

2007-01-09 Thread beast
I have these following data: a 100 a 102 c 100 a 102 b 111 c 100 c 102 c 100 c 100 a 102 ... I would like to have a list (either array or hash) with unique line . Any help would appreciated. Thanks. -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL

Re: remove duplicate lines

2007-01-09 Thread Mathew Snyder
beast wrote: I have these following data: a 100 a 102 c 100 a 102 b 111 c 100 c 102 c 100 c 100 a 102 ... I would like to have a list (either array or hash) with unique line . Any help would appreciated. Thanks. The way I would do it is to place the initial data instance

Re: remove duplicate lines

2007-01-09 Thread beast
Mathew Snyder wrote: The way I would do it is to place the initial data instance into a hash if it doesn't already exist. Just do a next if it does. It will remove duplicate key, not line. my %hash=(); while () { chomp; my ($key, $val) = split /,/; $hash{$key} = $val; } while ( my

Re: remove duplicate lines

2007-01-09 Thread Owen Cook
On Wed, Jan 10, 2007 at 12:47:25PM +0700, beast wrote: I have these following data: a 100 a 102 c 100 a 102 b 111 c 100 c 102 c 100 c 100 a 102 ... I would like to have a list (either array or hash) with unique line . Any help would appreciated. Thanks. 1. You need to read

Re: remove duplicate lines

2007-01-09 Thread beast
Owen Cook wrote: my %data_hash; while(DATA){ chomp; # Get rid of line feeds my @bits = split; $data_hash{$bits[0]} = $bits[1]; } It will only remove duplicate key. Is this still acceptable in perl (its very ugly =( while () { chomp; my ($k, $v) = split /,/; my $tmp_key

RE: remove duplicate lines

2005-06-13 Thread Tielman Koekemoer \(TNE\)
I'm also very new to Perl and wrote a long and newbyish script that does exactly what the Unix command sort FILENAME | uniq does just to see how it can be done. How long? Because you can do that on one line in perl. :-) Hi John (we're not worthy - Wayne's World) ;-) I've just come

RE: remove duplicate lines

2005-06-13 Thread Chris Devers
On Mon, 13 Jun 2005, Tielman Koekemoer (TNE) wrote: PS Will the Perl Cookbook have examples to common tools such as this? Yes. Oodles of them. What does everyone think of the book - very helpful? Extremely. -- Chris Devers -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional

RE: remove duplicate lines

2005-06-13 Thread Thomas Bätzler
Tielman Koekemoer (TNE) [EMAIL PROTECTED] asked: I've just come back to your post above (30/05/05) and I've tried to get the internal workings of the code to agree with my brain. Please help! perl -e'print sort grep !$seen{$_}++, ' FILENAME Initially I thought you were creating a

RE: remove duplicate lines

2005-06-13 Thread Chris Devers
On Mon, 13 Jun 2005, Thomas Bätzler wrote: It also used to be part of a Book/CD set called the Perl Bookshelf. AFAIK [_Perl Cookbook_ is] out of print but you might be able to pick it up at a used book store (or onlinle at ABEbooks) cheaply. The first edition went out of print, but the second

Re: remove duplicate lines

2005-06-13 Thread John W. Krahn
Chris Devers wrote: Or the forthcoming, more up-to-date _Perl Best Practices_: http://www.oreilly.com/catalog/perlbp/ Though in this case you'll have to wait until July or August for it to be released, but I have no doubt that it'll be worth the wait. Yes, anything by Dr. Conway will be

Re: remove duplicate lines

2005-06-13 Thread John W. Krahn
Tielman Koekemoer (TNE) wrote: I'm also very new to Perl and wrote a long and newbyish script that does exactly what the Unix command sort FILENAME | uniq does just to see how it can be done. How long? Because you can do that on one line in perl. :-) Hi John (we're not worthy -

RE: remove duplicate lines

2005-05-30 Thread Tielman Koekemoer \(TNE\)
Wow, I'm really confused. I'm trying to remove duplicate lines from a marc21 text file. I have spent countless hours searching for scripts etc. I'm also very new to Perl and wrote a long and newbyish script that does exactly what the Unix command sort FILENAME | uniq does just to see how

Re: remove duplicate lines

2005-05-30 Thread John W. Krahn
Tielman Koekemoer (TNE) wrote: Wow, I'm really confused. I'm trying to remove duplicate lines from a marc21 text file. I have spent countless hours searching for scripts etc. I'm also very new to Perl and wrote a long and newbyish script that does exactly what the Unix command sort

Re: remove duplicate lines

2005-05-30 Thread Jay Savage
On 5/30/05, John W. Krahn [EMAIL PROTECTED] wrote: Tielman Koekemoer (TNE) wrote: Wow, I'm really confused. I'm trying to remove duplicate lines from a marc21 text file. I have spent countless hours searching for scripts etc. I'm also very new to Perl and wrote a long and newbyish

Re: remove duplicate lines

2005-05-28 Thread Dale
Hi John Doe, you wrote : *snip* To learn perl, you have to spend hours and hours studying books/docs/examples, thinking, trying etc. Depends on your own individual learning style. Some folk would breeze through the perldocs. Personally, I feel that they read like stereo instructions.

remove duplicate lines

2005-05-27 Thread Jack Daniels (Butch)
Wow, I'm really confused. I'm trying to remove duplicate lines from a marc21 text file. I have spent countless hours searching for scripts etc. What I find frustrating while trying to learn Perl, is that most solutions assume you know what to do. For example, someone gives the code to find

Re: remove duplicate lines

2005-05-27 Thread John Doe
Am Freitag, 27. Mai 2005 13.56 schrieb Jack Daniels (Butch): Wow, I'm really confused. I'm trying to remove duplicate lines from a marc21 text file. I have spent countless hours searching for scripts etc. What I find frustrating while trying to learn Perl, is that most solutions assume you

Re: remove duplicate lines

2005-05-27 Thread John Doe
Hello As an addition to my last post: Am Freitag, 27. Mai 2005 13.56 schrieb Jack Daniels (Butch): Wow, I'm really confused. I'm trying to remove duplicate lines from a marc21 text file. I have spent countless hours searching for scripts etc. What I find frustrating while trying to learn

Re: remove duplicate lines

2005-05-27 Thread Eric Walker
duplicate lines from a marc21 text file. I have spent countless hours searching for scripts etc. What I find frustrating while trying to learn Perl, is that most solutions assume you know what to do. I think this is nothing special; if somebody has coded a _solution_, he (hopefully) knows

Re: remove duplicate lines

2005-05-27 Thread John Doe
Am Freitag, 27. Mai 2005 17.15 schrieb Eric Walker: Can't you run uniq from the command prompt to get rid of the duplicate lines? perlnewbie.. [...] Yes I can, $ cat marc21textfile | uniq outputfile but the OP explicitly wanted a perl script and states For now, I just want to remove

Re: remove duplicate lines

2005-05-27 Thread Eric Walker
On Friday 27 May 2005 01:22 pm, John Doe wrote: Am Freitag, 27. Mai 2005 17.15 schrieb Eric Walker: Can't you run uniq from the command prompt to get rid of the duplicate lines? perlnewbie.. [...] Yes I can, $ cat marc21textfile | uniq outputfile but the OP explicitly wanted a

[OT] Re: remove duplicate lines

2005-05-27 Thread John Doe
. thanks, joe Ok, the perlnewbie tag is for me, not you. Seems like you got upset. sorry for the misunderstanding.. perlnewbie Oups - I have to be sorry for my misunderstanding. Your solution is of course better for the problem how to remove duplicate lines. sorry again, joe

RE: remove duplicate lines (OT)

2005-05-27 Thread Bakken, Luke
$ cat marc21textfile | uniq outputfile Where's Randal for a UUOC award? -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] http://learn.perl.org/ http://learn.perl.org/first-response

Re: remove duplicate lines (OT)

2005-05-27 Thread John Doe
Am Samstag, 28. Mai 2005 00.06 schrieb Bakken, Luke: $ cat marc21textfile | uniq outputfile Where's Randal for a UUOC award? Ok, for all using perl for everything and not beeing experienced with cmdline tools as me: $ uniq marc21textfile outputfile Useless Use Of Cat (I read the INPUT

Re: remove duplicate lines

2002-10-04 Thread waytech
On Wed, 02 Oct 2002 04:17:29 -0400, Sudarshan Raghavan wrote: On Fri, 27 Sep 2002, waytech wrote: hi, i want to remove duplicate lines from one file(original file), and save the result to another file. the origianl file like

Re: remove duplicate lines

2002-10-04 Thread Sudarshan Raghavan
On Fri, 4 Oct 2002, waytech wrote: hi everyone, Thanks for all replys about my question. I know here are lots of perl experts. it's amazing that someone can short that long program into one line. I am a perl newbie, i wonder whether someone can explain what

Re: remove duplicate lines

2002-10-02 Thread jontn_swift
This is a classic case for my favorite one-liner: perl -e'while(){print unless $seen{$_}++}' infile outfile On Wed, 02 Oct 2002 04:17:29 -0400, Sudarshan Raghavan wrote: On Fri, 27 Sep 2002, waytech wrote: hi, i want to remove duplicate lines from one file(original file), and save

Re: remove duplicate lines

2002-10-02 Thread Sudarshan Raghavan
On Wed, 2 Oct 2002, jontn_swift wrote: This is a classic case for my favorite one-liner: perl -e'while(){print unless $seen{$_}++}' infile outfile You favourite one-liner can be even shorter perl -n -e 'print unless $seen{$_}++' infile outfile -- To unsubscribe, e-mail: [EMAIL PROTECTED]

Re: remove duplicate lines

2002-10-02 Thread John W. Krahn
Sudarshan Raghavan wrote: On Wed, 2 Oct 2002, jontn_swift wrote: This is a classic case for my favorite one-liner: perl -e'while(){print unless $seen{$_}++}' infile outfile You favourite one-liner can be even shorter perl -n -e 'print unless $seen{$_}++' infile outfile You want

Re: remove duplicate lines

2002-10-02 Thread John W. Krahn
John W. Krahn wrote: Sudarshan Raghavan wrote: On Wed, 2 Oct 2002, jontn_swift wrote: This is a classic case for my favorite one-liner: perl -e'while(){print unless $seen{$_}++}' infile outfile You favourite one-liner can be even shorter perl -n -e 'print unless

Re: remove duplicate lines

2002-10-02 Thread Sudarshan Raghavan
On Wed, 2 Oct 2002, John W. Krahn wrote: John W. Krahn wrote: Sudarshan Raghavan wrote: On Wed, 2 Oct 2002, jontn_swift wrote: This is a classic case for my favorite one-liner: perl -e'while(){print unless $seen{$_}++}' infile outfile You favourite one-liner can

Re: remove duplicate lines

2002-10-02 Thread Sudarshan Raghavan
On Wed, 2 Oct 2002, Sudarshan Raghavan wrote: On Wed, 2 Oct 2002, John W. Krahn wrote: John W. Krahn wrote: Sudarshan Raghavan wrote: On Wed, 2 Oct 2002, jontn_swift wrote: This is a classic case for my favorite one-liner: perl -e'while(){print unless

Re: remove duplicate lines

2002-10-02 Thread John W. Krahn
Sudarshan Raghavan wrote: On Wed, 2 Oct 2002, Sudarshan Raghavan wrote: On Wed, 2 Oct 2002, John W. Krahn wrote: John W. Krahn wrote: Sudarshan Raghavan wrote: On Wed, 2 Oct 2002, jontn_swift wrote: This is a classic case for my favorite one-liner:

Re: remove duplicate lines

2002-10-02 Thread John W. Krahn
John wrote: All, Hello, This is the first time I've ever seen a bunch of guys reaching for bragging rights on just how short theirs is...and to document it, too! g Then you need to check out Perl golf. :-) http://perlgolf.sourceforge.net/ John -- use Perl; program fulfillment -- To

Re: remove duplicate lines

2002-10-01 Thread Patricia Hinman
Sep 2002, waytech wrote: hi, i want to remove duplicate lines from one file(original file), and save the result to another file. the origianl file like this: [EMAIL PROTECTED

Re: remove duplicate lines

2002-10-01 Thread Sudarshan Raghavan
On Tue, 1 Oct 2002, Patricia Hinman wrote: I wanted to do this without completely loading the page into an array and then checking for duplicates. The only case the entire file is read into memory is when all the lines in the file are unique. But using a hash reduces the number of

remove duplicate lines

2002-09-30 Thread waytech
hi, i want to remove duplicate lines from one file(original file), and save the result to another file. the origianl file like this: [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL

Re: remove duplicate lines

2002-09-30 Thread Sudarshan Raghavan
On Fri, 27 Sep 2002, waytech wrote: hi, i want to remove duplicate lines from one file(original file), and save the result to another file. the origianl file like this: [EMAIL PROTECTED